Programmatically it'll make no difference whether you go from iPad to iPhone or vice versa. Graphically, though, you need to make sure that everything fits on an iPhone and isn't physically too small to where it's difficult to precisely touch the controls.

You also have two ways of scaling. You can scale the pixels so you're showing the same area of the game, just double/half the size, or you can keep it visually the same size but show double/half of what was visible before. You need to figure out what's appropriate for your game. Perhaps seeing more of the world makes the game too easy, perhaps it won't.

Whatever route you go, it's not a bad idea to try it on both devices when you make major decisions so that you don't find out it was a bad decision 4 months later when you try to make the entire version at once for the other device.