I'm looking for some advice on stocking a parts bin. I'm just getting started with DIY electronics, so I have no inventory to speak of.

Right now, I have some extra parts I picked up a while back (a few LM78/79 and LM317 voltage regulators), plus a stash of power inlets, fuse holders, switches, toroidals, electrolytic caps, and chip amp ICs for my first amp project.

Don't use carbon film resistors, they're the worst. In fact, they're the worst of two worlds: As film resistors, they are inductive due to their helical structure (like a ribbon wound on an cylinder). This also gives them a lower voltage rating than carbon composite resistors (these are useful, non inductive, high voltage rating due to continuous voltage drop across). The carbon has a high TC (which means high temp coeff, their value drifts with temp) and is noisier than metal film.
So either carbon composite (CC) or metal film. Most people prefer metal film: low TC, low noise, tight tolerances (1%) and much higher mechanical strength. People like Vishay Dale RN55D, CMF55...I once stocked up on the MRS25 series. There are also resistor kits where people sell you 1000 resistors for like 10$ on ebay or here in the swap meet section.

LEDs...get 20mA type, any color you like. Again, check ebay for kits (US or Hong Kong).
Diodes...get some beefy diodes too, like the 1N5407.
Caps...For audio use ceramic types are not too popular. Electrolytic caps below say 4u7 can can be replaced by film caps. Again, look for kits. Panasonic ECQP series caps (polypropylene and foil) are nice, so are Wima MKP10, MKP4 or FKP1. With electrolytic caps look for low ESR (effective series resistance) and 105° temp rating, like motherboard capacitors (no kidding, computer parts are awesome).
silicone...Get some BC550C and BC560C, these are generic low noise npn/pnp transistors that are quite common and cheap.

Regularly check the Swap Meet section, people are often selling batches of stuff.

That's great info -- thanks! I neglected to mention I have a dozen MUR860 diodes as well. I added the 5407s to my list since they're cheaper but will do nicely for less critical projects.

So, in between swap meets, where's a good place to find polypro caps? The Panasonic ECQ's are listed on Digikey, but are all non-stock items. There wasn't much else to choose from either. No Panasonics at Mouser either, but at least they have Wimas.

Polypros aren't THAT rare, are they? I don't understand why they're not as abundant as electrolytics and ceramics.