I tried to compare different mp3 decoders, in order to evaluate the benefits of dithering (MAD and foobar2000) and noise shaping (foobar2000 only), against simplier decoders, as LAME or Fraunhofer (Winamp 5.0). I didn't hope anything first. But I've finally succeed with one sample, and decided then to resume the test with others samples.I've wrote a complete review in french, but I can't translate it properly. In few words:

 For this test, I didn't reproduced real listening conditions. Difference are so subtle that I have to increase the level of my amp in order to maximise the audible difference. In order to make it possible without being completely deaf in few seconds, I've selected very quiet samples. With classical music, it's not really hard to find such samples (even full tracks are sometimes totally quiet). I didn't evaluate any decoders on pre-instrumental silence, but only when music was playing

Four decoders were tested:

 MAD, basically in order to evaluate the real performance of this mythic decoder, and technically in order to rate the benefits of dithering WITHOUT noise shaping. LAME, to mesure the difference between MAD and a simplier decoder. Fraunhofer: in order to evaluate possible difference with LAME (both encoders are really similar, so difference may not be audible or existent) foobar2000: to see if noise shaping could make a difference with MAD dithering.

MAD isn't impressive. On four samples, result was worse than simplest decoders: increase in noise, without audible benefit on other points of the signal (aliasing, higher definition, etc...). But on some other samples, real improvements were noticed (less aliasing, less naturel and less synthetic sound)

LAME and Fraunhofer (WA5) are close, but not totally identical. Both are noisy, but less than MAD (good point). With some samples, sound was irregular, unatural... MAD partailly correct this feeling, and foobar2000 was perfect in comparison. Something like aliasing or moiré was introduced by the decoding stage. Slight difference between LAME and Fraunhofer (hard to ABX : up to 30...50 trials), in favour of Fraunhofer, less synthetic and maybe slightly more noisy

foobar2000 : each time winner. The less noisy of all challengers on all samples. The only additionnal noise audible was a very high-pitched band, very typical, and audible on extreme conditions (near silence sound, and with my amp pump to the maximum). All aliasing, irregularities... were gone: instruments are smooth, naturel. Nothing synthetic at all. With one sample (the #7), the sound was clearly better than the reference file (CD!). The lack of dithering on mastering seem to be prejudicial on very low volume part, and foobar2000 decoding correct the aliasing present on the CD. Simply amazing conclusion....

P.S. It's not a MAD CHALLENGE. I repeat again that the listening conditions of the test are not reproductible on daily music listening. I've just put the decoder behind a magnifying glass, by using low volume samples.