I want to ask everyone. Before the test finish, which codec will win? Of course you may have to take a guess/bet or based on your experience/personal preference.

I have certain vague expectations about which codec will win, but I won't tell you (and I ask everyone else with knowledge about the codecs under test not to do so either) since this could influence the test.

A (IMHO) more important question: Igor, are all session results excluded for which some hidden reference is graded with less than 5.0? Maybe you should mention the post-screening procedure so that listeners behave carefully enough.

And about the resampling: Except for CELT, was the input or the output of the codecs resampled to 48 kHz?

A (IMHO) more important question: Igor, are all session results excluded for which some hidden reference is graded with less than 5.0? Maybe you should mention the post-screening procedure so that listeners behave carefully enough.

You're right. Here are the rules:

CODE

If at least one of the following errors takes place then the result will be INVALID.1. graded the reference lower than 5.02. didn't grade the low anchor.

If one listener will submit INVALID result then he will be informed and will have one more and unique posibility to submit result for one particular sample.Now if the listener will submit 3 or more invalid results then only ABX results will be accepted from him/her or directly rejected in abussive cases.

QUOTE (C.R.Helmrich @ Mar 23 2011, 17:26)

And about the resampling: Except for CELT, was the input or the output of the codecs resampled to 48 kHz?

The sample folders are included in the zip files. You must unzip the folders and decompress the FLAC files back to WAVE before they can be used in the test. The new filename extension must be .wav instead of .flac, otherwise the filenames must be unaltered. The folders must be placed in the ABC-HR folder:

You can have the FLAC files in the same folders if you don't want to delete them. They will just occupy some additional disk space (about 366 MB for all 30x6 samples).

Please don't abuse the bandwidth that is available for this Dropbox account. It makes no sense to download these versions if you can use the smaller original package (or packages) and let the bat file (or files) automatically process the test samples.

EDIT

If anyone else wants to host these FLAC versions I can provide also the individual sample packages and all samples in a single package (PM me).

Participants who don't want to worry too much about the grading rulescan simply ignore them. Listeners should do their best to rank the samples and be careful to identify the hidden references. Listeners should ABX teststhey are at all unsure.

1) If the low anchor is not graded, or if any hidden reference is graded below 4.5-5(*1a) (see App.) the result is INVALID.

*1a It will be discussed with people who have experience of conducting public tests when final results will be ready.

2) For each sample with a ranked reference or an ungraded low anchor thelistener will have a single chance to submit a replacement test run forthat sample. The replacement test must cover all codecs, not just thecodecs with the ranked reference. (This also covers cases where thereference is ranked but still at or above 4.5)

3) If a listener submits more than 2/15 (4 for 30 samples submitted) INVALIDresults then only ABX results will be accepted, or the listener will be excludedcompletely in cases of apparently abusive behavior.

App. These rules aren't extremely strict in order to allow for simple humanerror while still excluding careless participants.

A stricter procedure to exclude all ranked references risks a systemicbias against any codec which are very good on a few samples and thussubject to more reference confusion by causing those samples to be excludedand weighing the test towards other samples.

During the past tests we usually had 1) a pretest discussion in "Listening Tests", 2) a news thread in which the open test was announced and commented and finally 3) a news thread in which the results were published and commented.

This time we did not have a pretest discussion (at least not in a single dedicated thread). This thread is already the news & comments thread.

To: Admins/Mods

- Could you simply create a short (perhaps slightly edited) news article from the first post that would point to this thread?or- Could you move this thread to "Validated News", create a short (perhaps slightly edited) news article from the first post that would point to this thread and perhaps leave here, in "Listening Tests", a short note about the moved thread and a link to the new location?

Some people wait for the last day to send their results. That's ok. But it might be the case that someone can forget to send the results until 30th of March.So if You have already even a few results send them soon.

Of course, the last moment results will be accepted during 30th and 31st (~01.00 of GMT -3.00) of March.

As far as I can see all participants have received the answer. Thank You to All who participate.

I've had fun doing the tests but please make the samples shorter next time so that the participant can more easily establish points in the audio to focus on and avoid fatigue. For example sample 15 starts of with a female voice and then switches to a male voice, forces you to acquire a split personality

I'm extremely impressed by all four (non low anchor) codecs. Two in particular appear essentially transparent to my ear across the entire set of samples. I can only hope that those two are consistent and it's not a cycling of which is best in each case