Second, My apology to Roberto for bringing this issue from a different perspective. I really respect you for previous listening tests, taking initiative and creating momentum in the HA community for these kind of actions.

The first and foremost agenda behind creating Hydrogenaudio was to create a community who looks for the best quality audio without any compromise. Otherwise we could very well stayed within r3mix community. Where Roel thought his presets are "good enough" for majority of the audiences, and denied to move forward.

Honestly, I am not sure, from those tests whatever format wins, what is that going to prove other than knowing this particular format at this particular bitrate with this particular settings are better than other codec at this bitrate with these settings. Say that AAC won the test. We can only say that, yeah AAC has been improved and does better at this bitrate. But then what? Will AAC at 128kbps be recommended by Hydrogenaudio for the mass? If not, then why does it matter which format performs better at this bitrate? Don't you think that the average users are going to misinterpret this test and think that it is a recommended bitrate/encoder/setting.

If these tests are for non-audiophile users then we are doing a disservice by not making them aware that they shouldn't use any of these format at this bitrate, as they are not transparent. Again if it is targeted at the developers, I don't think this is the kind of test that they need.

Hydrogenaudio is famous for it's uncompromising stand on scientifically proven quality audio. If we keep on conducting listening tests again and again at 128kbps then average users are going to get a false message that it is now good to use newer codecs at this bitrate (otherwise why HA is giving that much importance on this bitrate?).

I don't think that just because some big companies like Apple are trying to push low quality DRM infected music to clueless people, it justifies giving any attention by an elite group like HA. Apple is actually harming the audio community by polluting average people's music collection with inferior quality music. This might in future help them getting more revenue by selling another round of same music at higher bitrate (this time with transparent quality). But from a consumer's point of view, why should we fall prey to their scam? I really hate it when Apple introduce HDD based player with as big as 10/20/30/40GB drives and try to shove down inferior quality music at 128kbps. What was their problem? Storage size wasn't an issue, newer format (AAC) is even more efficient. Why not give transparent music?

Heck, people at Hydrogenaudio were running after nothing less than transparent quality music at the age when storage size was really an issue (650-700MB CD-R based or usually 64-128MB flash player). Why then, at this age, HA community should give any hoot about 128kbps non-transparent music?

If we, as a consumer, don't demand higher quality then no industry is gonna give that. One example is that there are growing number of paid music sites are offering LAME -aps encoded files, just because of people's awareness and demand. I'm quite sure, Hydrogenaudio is, at a large, responsible for that awareness. The downside of giving much importance at Mid-bitrate (128kbps) is using this bitrate we are not achieving any goal. Neither transparency nor ability to stream at this bitrate.

It might be more useful to conduct some test at low bitrate (~64kbps) at which sound might not be transparent but will be vary useful for streaming media, or maybe even lower bitrate for voice communication.

If we really want some tests, It would be more useful to conduct a test at a bitrate where every codec reach transparency level. That will give a sense to the average user that how efficient one codec is against other and at what bitrate they become transparent. Granted, that type of test is not going to be very easy to conduct, but if done properly it will give very useful information.

Otherwise we are moving backward by giving more importance on the holy grail of 128kbps bitrate that big industry players are trying to shove down to our throat making us believe that this is the de facto standard of "quality audio".

The test will show how many listeners find each encoder transparent on each clip, and how big any "problems" are perceived to be.

The Hydrogenaudio "transparent encoding" recommendations will almost certainly remain unchanged. The 128kbps mp3 recommendations may change, and rightly so if this test reveals they're wrong or out-of-date.

This is useful information, and drives codec development. If no one was interested, no one would take part!

FWIW I find the tests fascinating - HA attracts people who are interested in audio coding, and though most are interested in transparent coding, just see the "state of the art" at any bitrate is very interesting.

I find it amazing just how close to transparent the best codecs at 128kbps are. This might not be the case in the mp3 test, but it certainly has been in multi-format tests.

HA has always concentrated on both ends of the scale:

a) what is the best sound that can be achieved at a given bitrate (even if that's a relatively low bitrate)b) what is needed to give "transparent" encoding in each format, andc) what samples cannot be transparently coded in each/any format.

Besides, HA is the sum of all its members, and it will go where ever the members take it. We've discussed everything from wax cylinders to DVD, 48kbps to 700kbps+. If there's something you're missing, just ask!

Cheers,David.

P.S. short answer: I don't think there's any danger of HA ever going entirely "low quality". For example, other currently open listening tests/challenges include DVD-A quality audio.

Roberto is doing a test at 128kbps because that's about as high as you can go and still have a reasonable chance that most people will hear a big enough difference to compare quality.

Listening tests at the audiophile level are nearly impossible to conduct. I challenge you to conduct one if you're interested. I tried a transcoding test where I wasn't terribly interested in the quality differences, just whether or not people can hear the difference if it came down to it. That turned out to be a whopping failure with a total of 6 results between 2 different users. Low anchor notwithstanding, all of the scores were above 4.3 (out of 5). A listening test at the levels you're talking about will produce scores even higher than that.

And the higher those scores are, the less meaningful they become, since your error margins become much larger.

I don't have much to add to 2BDecided's comment. Hydrogenaudio has never focused on only one thing, if the only interest of the HA community was transparent audio it would be a pretty boring place, especially not much has changed in this fiel since almost 2(!) years. There are many interesting sides of audio coding and HA is trying cover as many of them as possible in its usual scientific and in-depth manner, from meta-data over speech codecs to 24bit lossless audio.

dev0

--------------------

"To understand me, you'll have to swallow a world." Or maybe your words.

The Hydrogenaudio "transparent encoding" recommendations will almost certainly remain unchanged. The 128kbps mp3 recommendations may change, and rightly so if this test reveals they're wrong or out-of-date.

I think there is the issue, while transparent settings will remain unchanged 128kbps will not. So, today if I encode some file with today's recommended settings it is going to be outdated very soon (because rapidly developing modern codecs are not tuned yet). Do you think it should be advisable to recommend something that will be out of date soon, forcing the user to re-encode?

QUOTE

This is useful information, and drives codec development. If no one was interested, no one would take part!

I'm not saying there is not much interest. From the responses so far, I can tell there there are quite big interest in this test. I am just afraid that one after another 128kbps test will give false impression to the newbies that it is a preferred bitrate. (remember, some misinterpretation of the last 128kbps test result?).

I don't quite see how we're making the leap from what Roberto decides to do, to somehow having that equate to "Hydrogenaudio losing its direction." Seems that we get another one of these topics every other month or so (and most of them also try to tie in something that we need to be doing for some particularly upstanding reason, in this case not educating people about 128kbps because it "implies the wrong goals"), so I guess I'm not surprised to see this one.

Sorry, I just don't see it.

If Hydrogenaudio is "losing its direction" for some people, it's only because they have had too narrow a vision of this community.

Other than that, if you take issue with what you think Roberto is implying (i.e. that you think his continued 128kbps tests are giving the wrong impression of "what is good"), then it would be prudent to take that up with him instead of questioning whether this community as a whole is somehow degrading.

I really hate it when Apple introduce HDD based player with as big as 10/20/30/40GB drives and try to shove down inferior quality music at 128kbps. What was their problem? Storage size wasn't an issue, newer format (AAC) is even more efficient. Why not give transparent music?

What is not transparent for you might well be transparent to others.

In both tests I featured QuickTime at 128kbps, it was above the ITU transparency cutoff line. Sure, it won't be transparent at your silent acoustic room listening to it on Sennheisers. But when listening to your iPod, with earbuds, while jogging in a noisy park, I'm quite sure you wouldn't be able to tell the original from the encode.

Most people are perfectly happy with MP3 at 128kbps. So they can't complain about AAC at 128.

What I consider a real scam is people selling WMA 64kbps to their customers just because Microsoft says it is CD quality. WMA has been proven to be definitely bad quality at 64kbps. QuickTime has been proven to be quite close to transparency at 128kbps.

Regards;

Roberto.

--------------------

Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:http://www.rarewares.org

Other than that, if you take issue with what you think Roberto is implying (i.e. that you think his continued 128kbps tests are giving the wrong impression of "what is good"), then it would be prudent to take that up with him instead of questioning whether this community as a whole is somehow degrading.

This was exactly my meaning - repeating test of 128kbps will create confusion. But I agree with you that the title is a bit misleading (posted too hastily) . It should read "We" as the member instead of "Hydrogenaudio".

As i am not using music compression at all for my beloved music, and lossless compression is the worst i could think of for it, i find Roberto's 128 kbps listening tests extremely useful because this bitrate is a very good bitrate for movie encoding, which is in fact the only use of lossy audio compression for me .....

This was exactly my meaning - repeating test of 128kbps will create confusion. But I agree with you that the title is a bit misleading (posted too hastily) . It should read "We" as the member instead of "Hydrogenaudio".

Damn, there is no way to modify the title.

"Are we losing direction"?

The fact is that low bitrate area is still developing fast. So of course it is interesting to test the latest development. And as it has been said, 128kbps is close to the highest bitrate which still gives meaningful results from casual testers.

I wonder what would happen if we'd choose only people known to be experienced for high bitrate testing. Maybe other people would blame us for elitism or something, but that could be interesting.

Anyway, HA has never been only "high bitrate transparent quality" forum. Of course we seek for the highest quality, but we do that on all levels; the highest quality possible on all bitrates. We have have here speech codec developers as well as lossless codec developers and everything between, and in my opinion that's the way it should be.

That is the problem. We are running backwards. We are encouraging others to stay ignorant instead of what we could do to to help them.

QUOTE

Most people are perfectly happy with MP3 at 128kbps. So they can't complain about AAC at 128.

Most are happy with 128kbps Xing MP3. So, why bother testing and finding out which is better? What is your goal? Just to show that AAC is better than MP3? I don't see any advantage in showing them in the grand scheme.

QUOTE

What I consider a real scam is people selling WMA 64kbps to their customers just because Microsoft says it is CD quality. WMA has been proven to be definitely bad quality at 64kbps. QuickTime has been proven to be quite close to transparency at 128kbps.

There are scams, bigger scams or even real scams. But all are scams. Big industries trying to suck off the consumers to death. Are you saying that because they are less evil so they should be supported?

That is the problem. We are running backwards. We are encouraging others to stay ignorant instead of what we could do to to help them.

Do they even want your help? Ignorance is bliss.

QUOTE

Most are happy with 128kbps Xing MP3. So, why bother testing and finding out which is better? What is your goal? Just to show that AAC is better than MP3? I don't see any advantage in showing them in the grand scheme.

Why bother? Because there are people who care. Others don't care, and I won't go after them saying "U f00! You should be using codec XXX, it's better!"

When I announce test results, I don't write "now everybody should move to codec X because it's the best at this bitrate". I post results and people that care will switch codecs, people that don't care won't. Simple as that.

QUOTE

There are scams, bigger scams or even real scams. But all are scams. Big industries trying to suck off the consumers to death. Are you saying that because they are less evil so they should be supported?

Nope. I say that because I agree with what Apple does. IMO, they are selling high quality music. At any bitrate above that, people would start to whine the files are bigger than MP3 so AAC compression must suck. Then they would move on to the stores selling WMA at 64kbps.

On such large-scale business like the iTMS, you can't think only about the audio quality. Sure, lossless would be great. But then people would whine about outrageous download times, and Apple would spend shitloads of money on bandwidth, forcing them to raise the prices. You have to draw the line somewhere, and I personally agree with QuickTime AAC at 128kbps. People that are unhappy with this bitrate are free to go to a CD store and buy albums there.

This post has been edited by rjamorim: Dec 12 2003, 17:18

--------------------

Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:http://www.rarewares.org

hydrogenaudio is what its visitors make of it. If you don't like the goals of the listening tests being organized by somebody else (by the way, full credit should be given directly to Roberto), conduct your own which you think are better suited to your vision of what this place should be.

I wonder what would happen if we'd choose only people known to be experienced for high bitrate testing. Maybe other people would blame us for elitism or something, but that could be interesting.

HA is already being blamed for that. But there is a bright side of that, too.

QUOTE

Anyway, HA has never been only "high bitrate transparent quality" forum. Of course we seek for the highest quality, but we do that on all levels; the highest quality possible on all bitrates. We have have here speech codec developers as well as lossless codec developers and everything between, and in my opinion that's the way it should be.

The issue is not "why we can't test at certain bitrate?" My question is - what is our goal? As I mentioned in my original post, it would be more meaningful to see low bitrate performance of some codecs like HE-AAC, Vorbis, WMA or whatever a good candidate at that bitrate. In real life we can use it for streaming video or limiting movies to certain number of discs. Do you really see a lot of people using AAC/Vorbis/MPC at 128kbps for video in near future? For audio, as I said, it will serve no purpose - it is neither transparent nor we are facing storage limitations.

About facing storage limitations: We are if you consider Flash audio players, mobile phones and PDAs.

Last but not least, it might not be transparent, but people simply don't care and keep using it. Do a quick search on Kazaa, you'll find mostly 128kbps MP3s. If people cared about transparency (outside HA), we wouldn't have countless 128kbps MP3s floating around.

--------------------

Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:http://www.rarewares.org

The coin has two sides. If you want to go to a download vendor and say that you need one codec over another for a given format, or some particular format, or some minimum bitrate to be "CD quality" you will need some argument more convincing than "mo bettah." That would be comparable surveys of various coding schemes at variousbitrates showing what percentage of the participants really can tell original from compressed.

HA.org is not saying to start only encoding at 128kbps. In fact, 128kbps tests are done to say “hey we have these compression methods and we know they’re not perfect.” The tests conducted show obvious faults with certain test samples and certain encoders. This information in turn is very useful to programmers. I’m no expert on this site, only every once in awhile I put my two cents in. But I think this is an awesome place and every since I’ve been coming here I haven’t felt pressured to using a certain bit rate.

If people want to come here and read for like 5 minutes and leave thinking they’ve learned all they need to know because of seeing results of 1 listening test then they are fooling themselves. HA.org has never been about telling people what to do, other than forcing everyone to back up their claims with listening test results. But who can have a problem with that?

Furthermore 128kbps is probably a good bit rate to show faults with certain encoders without totally bottoming out. In other words 128kbps with many samples will sound good, but yet will still be distinguishable from the original. Which I think is most useful to programmers.

128kbps is also the most common bit rate for back-ups of dvds. If you are going to try to backup a movie you own to DIVX, you’ve not going to use 320kbps MP3 are you?

Why bother? Because there are people who care. Others don't care, and I won't go after them saying "U f00! You should be using codec XXX, it's better!"

When I announce test results, I don't write "now everybody should move to codec X because it's the best at this bitrate". I post results and people that care will switch codecs, people that don't care won't. Simple as that.

The problem with that, because of Hydrogenaudio's previous track record, people will take it as recommendation. I think, as a respectable member of this board, you could do a lot better than just "Idon't care about average people".

QUOTE

Nope. I say that because I agree with what Apple does. IMO, they are selling high quality music. At any bitrate above that, people would start to whine the files are bigger than MP3 so AAC compression must suck. Then they would move on to the stores selling WMA at 64kbps.

Here is my disagreement. Apple is polluting the audio community with Mid-Fi music just to create a user base. Do you gruntle when you stumble upon a 64kbps WMA file? Why don't you think people won't hate to see less than perfact music?

QUOTE

On such large-scale business like the iTMS, you can't think only about the audio quality. Sure, lossless would be great. But then people would whine about outrageous download times, and Apple would spend shitloads of money on bandwidth, forcing them to raise the prices. You have to draw the line somewhere, and I personally agree with QuickTime AAC at 128kbps.

Why? Lots of online music service doing business with MP3 -aps, or higher bitrate of other formats? Why is it so difficult only for Apple?

QUOTE

People that are unhappy with this bitrate are free to go to a CD store and buy albums there.

That is the problem. We are running backwards. We are encouraging others to stay ignorant instead of what we could do to to help them.

Do they even want your help? Ignorance is bliss.

Yup. That's one of the reasons I personally refuse to participate in a listening test.

It's not that I don't care, it's that I don't want to tune my ears to hear all the faults in the music I already have and in all the encoders that I might use. I want to enjoy music, not to listen to it and pick out all the flaws in it.

QUOTE

QUOTE

Most are happy with 128kbps Xing MP3. So, why bother testing and finding out which is better? What is your goal? Just to show that AAC is better than MP3? I don't see any advantage in showing them in the grand scheme.

Why bother? Because there are people who care. Others don't care....

And many more don't care but will care later.

That's what happened with me. I used to listen to the radio, my cassettes, etc. and never really cared how good it was. When I switched to cd's, sure it sounded better, but since they were cheap cd players, not substantially so. And I still listened to the radio, with all of it's faults.

Then I started listening to music more and more on my computer, with better speakers, headphones, etc., and I began to care. I began noticing that some stuff that used to sound okay didn't sound quite so good anymore.

I started out ripping my cd's at 128k because that was "cd quality." Everybody knew that because that's what people said! Afer a while I began noticing that it didn't sound quite that good and began using better settings. But I didn't really know which settings to use, so I just guessed.

Since my own ears are not tuned (because I don't want them to be!) and my hearing isn't perfect anyway, I have to depend upon competent others to run tests that produce useful results.

And as for other formats, although I don't use WMA (becaus I don't like Microsoft and don't want to do a closed format), if it hadn't been for places like HA, I might have actually believed them when they said WMA is cd at 64k, and when Nero was saying HE-AAC was cd at 48k, and so on.

And I don't use itunes, so I can't really get into the 128k AAC debate, but I can say that if I bought music online like that (any format), I would want it to be good enough to sound good now and still sound good in a couple years if my ears improve. I paid for it, it should be decent quality.

What I can't understand is why CBR is/was being considered for some or all codecs. That is one legacy which serves no purpose other than for streaming or other 'niche' areas such as video. I think that gives off the wrong message more than picking 128k as the bitrate.

The problem with that, because of Hydrogenaudio's previous track record, people will take it as recommendation. I think, as a respectable member of this board, you could do a lot better than just "Idon't care about average people".

Great. Now you are twisting my words.

"Idon't care about average people" <- I do. If I cared about HA only, I would be only conducing useless high-bitrate tests where you can get no usable outcome due to the lack of listeners. But I NEVER recommended anyone to use this or that when I presented test results. If someone is reading that into what I write, he lacks common sense.

QUOTE

Here is my disagreement. Apple is polluting the audio community with Mid-Fi music just to create a user base. Do you gruntle when you stumble upon a 64kbps WMA file? Why don't you think people won't hate to see less than perfact music?

Less than perfect for you, goddamit. Don't try to generalize everyone to your level.

And I already told you why they can't offer perfect music (lossless). If you choose to ignore my arguments, so be it.

QUOTE

And I thought the world is moving away from CD media eventually.

Go buy DVD-A then. Or SACD.

--------------------

Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:http://www.rarewares.org

Are you suggesting that HA members conclude from those listening tests that the formats are transparent? I surely hope that HA members don't conclude from tests where a difference is shown that anything is transparent.... How can they?

Also please let me know how you would conduct listening tests at transparent level. What would you compare? What would be the goal?

I'll remember that when I've burnt some CD-Rs full of ~120-140kbps mp3s, and I'm listening to them in the car.

I'm not implying that you must use -aps or something. But look at the flip side, if you had everything at -aps settings then the same disk you could use in your car, in your home theater system, in your friend's audiophile gears or just directly transfer to your portable system without transcoding (assuming that you wanna play in all those players). Now, not for doing that, you have to keep miltiple copies of same music in different media? I don't think this will be more cost effective or convenient.

QUOTE

Just because you don't use these bitrates, doesn't mean that no one does. If you don't like this thread/test, don't participate.

It's not because, I don't use it, so I don't wanna see this. The issue is that if we encourage with it, low quality musics will be floating around soon, exactly same as what we have seen with MP3 128kbps.

QUOTE

And let's be bluntly honest: if we don't talk about this low bitrate stuff, what are we going to talk about? Musepack -q5 still transparent for most samples?

I think the impact of conducting some low bitrate listening test over and over just to prove some codec's achievement is very big. I don't see any issue discussing about any bitrate.

QUOTE

So it's like any discussion - dip into the parts that interest you, and ignore those that don't - and don't complain because not everyone is talking about something that interests you!

For a regular discussion, I would just ignore this. But seeing a bigger impact of listening test result, I am just concerned about the misinterpretation.