When buying a first guitar the first thing to consider is what you want to play and what you listen to. If you like jazz or softer sounds you might want to by a hollow or a semi-hollow body guitar. If you want to play rock, getting a guitar with two humbuckers is probably an ideal choice. If you have no idea, you can't go wrong with a stratocaster clone, with three single coils. If you're more of a folk fan, or a singer/songwriter type, consider an acoustic guitar. And if you're one of the brave few that wish to delve into the world of classical guitar get a nylon string acoustic
A Hollow-body is, as the name suggests a guitar with a hollow body. These produce a warm, and round tone ideal for jazz and styles of music that don't require distortion. The main draw backs to these guitars are the fact that they don't handle effects and distortion as well as solid body guitars, and due to their bulky design can be cumbersome in the hands of a beginner guitarist. A semi hollow will alleviate some of these problems by being thinner, and having some wood surrounding the bridge and pickups, however you do lose some of that great hollow body softness.
Likewise, a solid body guitar is a guitar with a solid body. At entry level guitars of this type come in several varieties. Guitars with tremolos, guitars without tremolos, guitars with single coils and guitars with humbuckers. Avoid guitars with tremolos at the entry level, because they tend to having tuning stability problems which can cause alot of frustration. When it comes to the choice between single coils and humbuckers its not as cut and dry. A single coil pickup takes up a smaller lot on the guitar, so in most cases thats how you can tell the difference at sight. They are more susceptible to feedback (buzz and hum) than humbuckers but allow more frequencies and overtones into your tone. This means you can not use as many effects or as much distortion with a single coil equipped guitar. Humbuckers on the other hand cancel most feedback, and can handle the effects and distortion. These pickups are approximately twice the size of a single coil at sight.
With acoustic guitars and nylon string guitars handling feed back isn't a problem, however in buying these I reccomend getting one with out built in electronics as those adversely effect the tone and sustain of the guitar.
Now that you have an idea as to what type of guitar you would like, its time to go to the store.
Your first guitar should NEVER cost you more than $500. At this point in your career the most important thing is that its playable, doesn't have any hardware issues, and is reliable. All the niceties of more expensive guitars are irrelevant at this stage (locking tremolo, premium pickups, higher quality wood, neck-thru construction, etc.). For all you know, you might not even like guitar, and going into the store and buying that real nice $3,000 Steve Vai signature guitar so that it can become a wall decoration would be a tremendous waste. Furthermore, until you have facility on the instrument, you won't be able to notice or make use of the subtle nuances of a premium high end guitars tone.
As far as amps go, either buy a small non tube practice amp, are a small modeling amp. Buying the small practice amp will be cheaper and give you what you need. Amplification, and a few knobs to control EQ, and a little bit of distortion. The modelling amp on the other hand, will be more pricey, but will give you tons of effects to play with which, are an important part of the modern electric guitar.
Once you've picked out your guitar and amp, on your way to the register, pick up a beginner guitar method book. A little bit of work in one of these goes a long long way. Also make sure to ask about warranty and return policy. Its real easy to mistreat your first guitar, and you might have to get it fixed or you might decide you don't like guitar and want to return it. Last but not least, put the receipt in a safe place.
For those of you who decide you like guitar, ROCK ON!
Tips:

avoid guitars with bolt on mahogany necks, those necks can develop serious problems

your first guitar should never ever ever have a locking tremolo

avoid guitars with more flamboyant and less practical shapes (V, Star, K, anything resembling a polygon or medieval weapon)

don't be afraid to try out a jump start kit, although you can surely find better stuff, there's nothing wrong with going that route

If you have a friend that plays/knows something about guitar, definately consult them or even bring them to the store.

When at the store, don't be intimidated by guitar jocks demonstrating their chops (technique), they're just lonely and want attention.

Actually, shapes like V's and K's can be very ergonomic and help you develop good habits. The BC rich warlock, for example, fits perfectly over your thigh so you can have better upper fret access and your lead chops can improve greatly from that.

I actually would never advise a stratocaster (clone) as a first instrument. The reason is the whammy bar. It makes tuning a lot more time consuming and difficult. If you change strings, not knowing anything about gauges, and you put a heavier gage on, the bridge will tilt heavily forward. Enough to put beginners in panic mode. I think this can all be very discouraging for beginners

I searched a lot of buying guides on the internet before I bought my first guitar but I'd never came across this article! I bought a used Ibanez RG2550z with tremolo bridge, I really like it and it doesn't have whammy bar so the staff told me it won't be a problem, but I think it will be a pain in the arse to change strings... am I right? (still quite panic)