No not really. Many of us do it when we change cars but if somebody knocks your car their insurance may not pay out either way. Anyway my wife and i pay peanuts to insure our new car so it is not worth taking the chance unless you need to take out a mortgate to insure your car!

If you buy the car with tax on it, it will be taxed in the other persons name which means he still owns the car. (example) If you were to take it out and be caught on a speed camera breaking the limit he would be the one that gets the fine. You need insurance to tax a car in your name.
But the straight answer is yes you can park it on the street but if anything happens it will go to the past owner not you.
( Lets hope hes not big and gives you a run of kicking!!)

Just make sure the handbrake is working & the fueltank is empty ... you don't want it rolling away into the path of a coach load of people, causing the coach to swerve & crash into a petrol station which explodes hurling burning debris into all the surrounding houses & polluting the local river, killing all the fish for 50 miles downstream.