A new keypair is generated for each receiving address (with newer HD wallets, this is done deterministically).
The public key and their associated private keys (or the seed needed to generate them) are stored in the wallet data file.
This is the only file users should need to backup.
A "send" transaction to a specific Bitcoin address requires that the corresponding wallet knows the private key implementing it.
This has the implication that if you create an address and receive coins to that address, then restore the wallet from an earlier backup, before the address was generated, then the coins received with that address are lost; this is not an issue for HD wallets where all addresses are generated from a single seed.
Addresses are added to an address key pool prior to being used for receiving coins. If you lose your wallet entirely, all of your coins are lost and can never be recovered.

Bitcoin allows you to create as many addresses as you want, and use a new one for every transaction.
There is no "master address": the "Your Bitcoin address" area in some wallet UIs has no special importance.
It's only there for your convenience, and it should change automatically when used.

Bitcoin addresses contain a built-in check code, so it's generally not possible to send Bitcoins to a mistyped address. However, if the address is well-formed but no one owns it (or the owner lost their wallet.dat), any coins sent to that address will be lost forever.

Hash values and the checksum data are converted to an alpha-numeric representation using a custom scheme: the Base58Check encoding scheme. Under Base58Check, addresses can contain all alphanumeric characters except 0, O, I, and l. Normal addresses currently always start with 1 (addresses from script hashes use 3), though this might change in a future version. Testnet addresses usually start with m or n. Mainline addresses can be 25-34 characters in length, and testnet addresses can be 26-34 characters in length. Most addresses are 33 or 34 characters long.

Collisions (lack thereof)

Since Bitcoin addresses are basically random numbers, it is possible, although extremely unlikely, for two people to independently generate the same address. This is called a collision. If this happens, then both the original owner of the address and the colliding owner could spend money sent to that address. It would not be possible for the colliding person to spend the original owner's entire wallet (or vice versa). If you were to intentionally try to make a collision, it would currently take 2^107 times longer to generate a colliding Bitcoin address than to generate a block. As long as the signing and hashing algorithms remain cryptographically strong, it will likely always be more profitable to collect generations and transaction fees than to try to create collisions, as demonstrated by projects like the Large Bitcoin Collider which attempt to generate address collisions.

It is more likely that the Earth is destroyed in the next 5 seconds, than that a collision occur in the next millenium.