Should we have a stickied post/link about blind testing?

I was wondering if we should have a stickied post/link to put with the glossaries of terms in the Useful Links section. I'm a strong proponent of blind testing and using more objective measures for audio comparisons, but it seems like a lot of people do blind testing incorrectly and concluding false things as a result. The Sound Science forum shouldn't have so many incorrectly done blind tests.

As mikeaj mentioned, level matching is one important consideration to make. I'll admit I don't know everything about what is required for a useful blind test, but I know what things certainly bias one. The "proper way" would be to ideally consider all potential biases, e.g. volume differences, headphone/equipment appearances, knowledge about price, and construct the test in such a way that the subjects are not exposed to them.

If the lossy encoding involves lowpass filtering and there's significant energy in the higher frequencies that are being rejected, then there could easily be a change in volume. There could be other things going on as well, but this is the most obvious as far as I can see.

If the lossy encoding involves lowpass filtering and there's significant energy in the higher frequencies that are being rejected, then there could easily be a change in volume. There could be other things going on as well, but this is the most obvious as far as I can see.

to me thats also the difference to test for, as its the result of the lossy encoding

if replaygain is applied, the averaged volume difference will be spread all over the song instead of just those parts with rejected high frequencies, which doesn't seems to make things better to me either.

...and also apply gain to volume won't revive the rejected high frequncies, it's just applied on the low-passed frequencies isn't it?

I seem to have been answering practical questions as if they were hypothetical. As far as I can tell, you probably wouldn't want to level match lossy vs. lossless. You want the lower frequencies that have not been filtered out, to be the same volume more or less as the original, which should be what you get if you don't do any normalization.