The differences in sound you get from different MP3 players will pale in comparison to the differences in sound from using different earphones, and the quality (bitrate) of the actual MP3 song. While my preference lies with Etymotic earphones, there are literally dozens of different brands and models at wildly different prices out there, all with their pros and cons. Here are a few sites that can start you out on some research:

I recently got an Cowon Iaudio7 with 8Gb http://www.cowonamerica.com/products/iaudio/7/
I chose it because of excellent sound-quality, ability to play Ogg format and mostly, does NOT depend on some proprietry DRM format and because it behaves like a normal external USB-drive when connected to the PC, so you can just drag/drop files into it.

If ALL of these things don't matter to you and you don't mind being locked in by a monopolistic company then buy an iPod.

Any MP3 Player that has a customizable equalizer will give you great sound.

I may be very outdated but I really enjoy my IRiver IFP-790. It is no IPod that carries thousands of sounds but it has enough capacity to carry all I need since I mostly use it for when I am in the gym. No point of carrying an expensive toy with a high probability of destroying it in a rough environment. As for headphones I like the Sony Sportsclip since it is water resistant and it hooks to my ears so it doesn't fall out during my time on the treadmill or whatever cardio I am doing.

Which MP3 player has the best sound? Is it worth it to upgrade the earbuds (or headphones) and if so, what model would you suggest?
Thanks muchly

More...

+1 for what macal said. A good pair of IEMs (in-ear monitors) will do wonders for your listening experience. You should look for IEMs that use balanced armature drivers - imo they have excellent overall response and beat all but the most expensive moving coil drivers (these are essentially like tiny mini-speakers within the earbud and act in the same way that a normal speaker does). A stock mp3 player doesn't really have the kind of amplifier that will give you optimal performance, but the difference between the stock earbuds and these will still blow you away.

The only other thing is that my iPod Nano is the first piece of Apple gear I have purchased in over a decade - and it's a piece of crap. It's an 8 GB Nano. It simply never worked properly. It either freezes up once or twice a day or starts to behave really strangely in other ways. I took it to an iPod shop and they said that if I have to replace the main 'logic board' (their term) it's more than the thing cost. They could not tell me if the various problems I'm seeing are software or hardware. The problems really started in earnest right around the time the warranty expired.