I am afraid that a major "overhaul" of the listening test page and the instruction is not going to be ready in time for this test. I am planning to change the listening test pages and make them somewhat more attractive, but 1. I am not really talented in designing so I have to see who can help me and 2. this is going to take some time so it will be only available for the next teast at earliest. I hope to have less ranked references in this test. Most of the people who perform listening tests are HA readers anyways so I was actually assuming that they know perform a "correct" test.

Here is the updated bitrate table that also contains the encoding speeds (except for iTunes that has to be tested once the fixed version is out).I didn't want to redo all tests on the single-core machine and comparing iTunes' single-core results to the dual-core results of the other encoders is not fair (at least if any of the other encoders makes use of multi-threading).

1) These results are only an approximate value and were obtained on a dual-core Intel E6550 with 2 GB RAM running on Windows Vista SP1 32-bit2) Encoding speed will be measured with iTunes once the bug-fixed version was released. The buggy version encoded at 18x but produced lower bitrates.

Edit 1: IMPORTANT: Please notice that the low anchor wasn't taken into consideration for building the min, max, delta and average bitrates.Edit 2: Sorry for breaking the layout - I used a codebox so I don't exactly know what is going wrong here.

Sorry to flutter the dovecotes perhaps, but wouldn't it be fairer to rename the test into 140k then, in stead of 128k?

Short test samples do not represent the average bitrate behavior of complete tracks of various genres in a media library. In my tests the overall average was 127.8 kbps. I think it is close enough.

Since this thread is huge and things can easily get lost in it I'll replicate my test results here (once again):

QUOTE (Alex B @ Aug 20 2008, 17:44)

I finally had time to continue my bitrate tests with classsical music as I promised earlier in this thread (a thing called summer got in the way...)

After browsing through my lossless classical library I picked 25 "reference" tracks that should be quite representative. I avoided the extremely low and high lossless bitrates and tried to select tracks that have quite varied qualities.

Apparently iTunes has changed radically since my last test. Back then the 128 kbps VBR setting was suitable, but the 7.7 version uses bitrates in a more relaxed way and the 128 kbps VBR setting produces higher bitrates than before. Fortunately the 112 kbps VBR setting appears to be suitable for our test.EDIT: As explained earlier in this thread, the fundamental difference in the bitrate behavior was found out to be caused by a bug in iTunes. However, the "iTunes 7.7" bitrates in my test are now assumed to be correct. Since my test Apple has released iTunes 8.0, but apparently its MP3 encoder has not changed. iTunes 7.7 and 8.0 create identical MP3 files.

In addition, I retested the "various" bitrates with the latest encoder versions when applicable.

Since I didn't have the old iTunes version installed I couldn't test the "classical" bitrates with it (which would have been unnecessary anyway).

FhG, iTunes and LAME 3.97 have only one suitable VBR setting for this test so I'd suggest to calculate the average bitrate of these encoders and adjust the Helix and LAME 3.98 settings to match this average. Helix -V60 and LAME -V5.7 appear to be pretty close to this average with my test tracks.

Here are the new results:

Summary

Various - table and chart

Classical - table and chart

EDIT

I forgot to mention that if anyone wants to test FhG's bitrate behavior the bitrates must be measured correctly. Most programs don't show accurate bitrate values because FhG doesn't write Xing headers to VBR files.

Linux users are asked to use Wine with "wine wcmd /c DecodeXX.bat" from the "bin" directory.

Apparently "wcmd" was renamed to "cmd" in wine 0.9.21 which was released 2 years ago.

Is there any description of the samples available? E.g. if I want to test a specific genre I'd have to download all samples. Also, if one likes a track, it might be interesting what's the artist/title.

Thanks for the wmcd feedback. I will edit the readmes once I get home this evening.

As for the sample names - they will be published only after the test. With past tests I had the problem, that a lot of people focused on several samples for which I then had lots of results while other samples remained untested.