The Problem with A-B'ing and Why Neil Young is Right about Sound Quality.

With news of Neil Young pushing to get studio quality audio out to the consumer, it occurred to me that it would be good to get some perspective on how we go about comparing audio quality. One of the most problematic issues surrounding audio quality and file resolution these days has been that, more and more, we hear about people not being able to tell the difference between compressed audio and full-resolution audio, and by extension some people are claiming that efforts like Neil Young's are out of touch and irrelevant. I couldn't disagree more.

In my opinion, one of the biggest pitfalls in this arena is the dreaded A-B test. In our seemingly infinite desire to manipulate the response of our fellow humans, we tend to stage A-B tests in two ways:

1) Almost-Barely: A-B because you can barely tell, and thus proving that the two items are close enough to be interchangable.

2) Absolutely-Boldy: A-B because you want to show how different two things are (and typically the superiority of one of them, which the test-designer is likely selling).

In my opinion, both types of A-B tests won't tells us what we want to know, but it's the first type, the Almost-Barely tests, that I take issue with here. (And I'm also inadvertently and respectfully drawing into question what, Ethan Winer, (who I adore) has said about "scientific" A-B testing in Tape-Op #88, p.66.)

The Almost-Barely tests seem so objective. Here's A, and here's B. See, soooo close! The conclusion: because most people can't tell the difference, there's not really a difference. And then you stack up the stats on these results, and suddenly we're doing science.

The problem is that these tests assume that because two things are close enough in a quick test that the difference will also be indistinguishable over long stretches of time. However, this assumption totally misses how it is that we tend to actually experience things in our very real lives.

For example, we hear people talk about how one can't make out the difference between a hi-res MP3 and a 24bit WAV file (assumedly a difference similar to the one Neil Young feels is worth fighting for). Admittedly a hi-res MP3 and a 24bit WAV are relatively close enough in resolution that many people will not be able to pick them out in an A-B test.

But, we don't live with music like that. If your'e anything like me, you listen to a lot of music in a lot of styles and - over the course of, say, a month - perhaps you've absorbed well over a hundred listening hours across many different albums on a few different playback systems.

How can flip flip flip replicate what it is to live with that much music for that long? How can a drop of water emulate what it is to swim for hours?

If you want to do a real test of the differences, give people a music collection that's all MP3s for a month, then give them that same collection as 24bit WAVs for a month, and then ask which one's which, and I bet you will start to get some correct answers.

Why? Part of the answer is that, if given enough time, subtle differences will reveal themselves to us. Subconsciously at first, and eventually consciously, we become aware of new details, subtleties, nuances. We humans need time to truly come to perceive things in full detail. But details, once revealed, become important features in the big picture.

For example, when I started living with my partner I introduced her to what I call "good coffee." At first she kind of shrugged it off as my snobbery at work, and she couldn't really taste the difference. But then, after months of drinking the good stuff, she found herself to be a bit of a coffee snob, too. She could taste the difference because she had, simply, spent time with the good stuff. The coffee revealed itself to her, slowly and subtly. Her palate developed. And the thing about good coffee is that it holds more detail, nuance and, therefore, interest.

But it takes a while to become aware of that depth and complexity. Had she done a flip-flip-flip A-B and made her choice to only drink the cheaper stuff because, "you know, they're basically the same," she'd have missed an opportunity to develop her palate.

I think the same thing can be said for the resolution of music, and it breaks my sonic heart to think of the A-B tests out there designed to convince someone that because they can't tell the difference today they won't tell the difference in a month or a year. A-B tests may be designed to show that subtle differences don't matter, but what they really do is shut down the possibility that those subtle differences could be the key to someone's aesthetic awakening.

If you can't tell the difference when you flip flip flip between two subtly different sounds, please know that I'm here cheerleading for you to slow down and be a real human being. Live with the better quality for a while, and see if over time you too start to hear things you never heard before. There is nothing like real life to truly test the quality of something.

So, for this reason, I herald Neil Young (as I always have, come to think of it) for sticking to his beliefs and making efforts to bring affordable and easily accessible hi-resolution audio to the market. It's time we left the MP3 behind and opened back up the possibility of people developing their ears and loving great sound once more. Who knows where such high standards could lead us.

It's also worth noting that Neil Young seems to be proposing 24bit, 192k files on new-generation pocket-sized playback devices. Give anyone 192k files, a decent D-A converter, a robust headphone amp and some great ear buds, and then put them back on an MP3/iPhone rig and I think we might not need a month to tell the difference. This also supports Mr. Young's claims that an MP3 only contains 5% of the information found on a professional analog recording. Neil Young seems to be proposing a far greater leap than the short gap between 44.1k WAVs to AAC files.

"My goal is to try to rescue the artform that I've been practicing for the past fifty years." - Neil Young

4-11-12: I just came across this statement from Neil Young's website. I hadn't seen it before, but thought I'd share it here. Digging the part about "senses opening up."

"The spirituality and soul of music is truly found when the sound engulfs you and that is just what 2012 will bring. It is a physical thing, a relief that you feel when you finally hear music the way artists and producers did when they created it in the studio. The sound engulfs you and your senses open up allowing you to truly feel the deep emotion in the music of some of our finest artists….This is what recording companies were born to give you and in 2012 they will deliver." - Neil Young, May 2011

Tape Op is a bi-monthly magazine devoted to the art of record making.

Sponsored

Sponsored

More Entries

Remember Nino from issue #67? His new Bird and Egg Recording Studio is hosting "an open house of sorts. On Sunday, October 18th between the hours of 2pm and 6pm Bird and Egg will be open for you to see and feel. There will be wine and hors d'ouvres...

Simultaneously making apparent the depths of both my dorkiness and my loyalty, I steadfastly stood by Imogen Heap's "Hide and Seek" all through its various tribulations over the last couple years -- first with its questionable placement on the OC,...

I just got an email from a company telling me that I need to buy the new version of a product of theirs - I already own the one that came out several years ago. I guess the $3000 I invested in their hardware wasn't enough, and now I need to buy a new...

Ina previous blog post, we saw the beginnings of this project. Damian Wagner was in issue 64 - here's a crazy project where he's amplifying the sound of the earth! "After destroying 3 subwoofers and having to completely redesign the
playback engine I...

Michael Cozzi (Guitarist/Producer Shriekback, Sky Cries Mary etc... ) dropped us a line about his newly built Moscozzi Studio in Seattle. he says, "I was able to complete it on a shoestring budget using largely re-salvaged materials." Luckily for...

If you want to stay in business you need to record all kinds of bands, including those you don't care for. (This is actually a shock to some people. It is. They don't last long, and their refusal to work with anyone except bands they hand-pick is...

Your magazine was not the first place I'd seen Count's "I Have a Credit Problem" essay [Tape Op #89], but I feel compelled to respond.
I agree with his general ideas - credits should be shown, and the current "album experience" in the digital...

(me and Elliott rolling in Jackpot!'s original MCI JH-16, Feb. 1997)
As most readers of Tape Op may have picked up on by now, I have been involved in the archiving and cataloging of the music of my late friend, Elliott Smith. Here's an interview I...