Judging Hi-Fi

Why listening to the tune is a fool-proof method of evaluating hi-fi
components

The A/B demonstration has long been the standard method of evaluation of
hi-fi components. You listen to one component, then listen to a second
component, and buy the one that sounds best. This sounds simple enough; but how
do you determine which component sounds the best? Some people listen for
distortion, or even break the sound down into component parts and evaluate the
high-end, mid-range and low-end. Others look (or listen) for sound stage, or
depth. Many just give up on it all and rely on the reviews and specifications
sheets. Our approach has always been to "listen to the tune". While this sounds
almost too simplistic to be useful, a careful examination of this technique will
reveal that, not only is it surprisingly accurate, it is based on a very solid
foundation.

Problem #1: Distortion

How many times have you heard someone comment about hearing distortion from a
hi-fi system? The truth is most people do not fully understand the concept of
distortion. You do not hear distortion! You may very well hear a bad sound
coming out of a hi-fi. However, the distortion is simply the difference between
the good signal that went in at the front-end of the system and the bad sound
that is coming out at the speaker end. Since you are rarely in a position to
know exactly what was on the record, you can not be sure how the signal has been
changed. Thus it is very risky to try to evaluate a hi-fi by trying to identify
distortions.

Scientific distortion measurements can be made. However, these are usually
based on tests using very simple signals and, like most specifications, are not
a good indication of the performance a component will deliver when called upon
to handle a complex musical signal. Measurements are also very limited in their
scope. They only measure certain specific types of distortion. In reality,
anything that changes the musical signal is a type of distortion whether anyone
has figured out a way to measure it or not.

There is only one thing you need to remember about distortion and its effect
on the musical signal:

A musical signal, at any given instant, can be described by an amplitude
(loudness) and a frequency (pitch). Thus, distortion, by definition, always
changes either an amplitude or a frequency or both. The significance of this
will become clear very soon.

Problem #2: More Distortion

Problem number two is really just more of problem number one (distortion). We
have just pointed out that it is very difficult to evaluate the distortion of a
component because you have very little information about the original signal.
This same limitation haunts other methods of evaluation as well. Breaking the
signal down into the component parts; i.e., highs, mids, and lows, rarely works
because you end up making judgments based on quantity rather than quality. You
may very well end up picking the system that gives you the most bass. It's
important to remember, you do not know "how much" bass was on the CD/record.

A very attractive way to evaluate a hi-fi is to sit back, close your eyes,
relax and picture the performers in your mind. You can almost feel them there in
the room. You could actually point to them. As attractive as this scenario may
seem, it suffers from the very same limitations as the other methods. You do not
actually know where that particular performer was standing when the recording
was made. In fact, in many studio recordings, the "position" of a performer was
simply determined by the position of a pan pot or balance control on a recording
console! In truth, do you really care where he was standing? Most people would
rather listen to a good pianist in the next room than a bad piano player eight
feet in front of them and 24.7 inches to the left. Don't get us wrong. The
"image" can be a very attractive aspect of a hi-fi system's performance.
However, it is only of concern after you have sorted out the musical aspects of
the system.

The Grand Solution

Now that we have spent the better part of a page explaining the limitations
in hi-fi evaluations, we feel compelled to offer a solution. If we are to make
an evaluation based on listening to music, we are forced to ask ourselves, "What
is on the CD/record?" The truth is, we know very little about the content of the
record. We don't know how much bass there is, or where the vocalist is standing,
or how loud that little tinkle of the triangle should be (you remember -- the
one, half way through the second cut on side B). In fact, we know almost nothing
about the original signal on that record. Whether we like it or not, this forces
us into making assumptions. In our view there is only one set of assumptions
that you can fairly make. There are professional musicians on that CD/record.
They must be good or they wouldn't be making an album. Ergo, there is music on
that album. (The validity of our second assumption could be questioned in some
cases.)

If you ask yourself musical questions when evaluating hi-fi components, you
have a much better chance of getting things right. Rather than evaluating the
sound of the hi-fi, evaluate the performance of the musicians. Anything a hi-fi
does to degrade the sound must lower the perceived quality of the musical
performance. You can ask questions like, "Is there emotion in the vocalist's
voice?" "Are they having a good time, or are they just playing because they are
getting paid for it?" This method of evaluation works even though you do not
know how good the original performance was! A bad hi-fi can degrade the musical
performance. However, a good hi-fi cannot improve upon the original signal on
the CD/record (it can only reproduce it as faithfully as possible).

Thus the simple fact that the performance sounds more musical on system A
than it does on system B, gives a clear indication of the minimum level of
musical quality present on the CD/record. While we can not know if system A is
recovering all the information on the CD/record, we certainly know that system B
is not.

Enter the Tune

Our favorite method of evaluating the musical performance, and thus the
performance of the hi-fi, is simply listening to the tune. Many people
immediately dismiss this as being obviously too simplistic to provide meaningful
results. But, in actual practice, this is an all-encompassing technique that
more clearly brings out differences in hi-fi systems than any other method of
evaluation we have ever used. The music on a CD/record consists of a signal
that, at any instant, can be described with two parameters, frequency and
amplitude (or pitch and loudness, if you prefer).

Any type of distortion, whether or not we can measure it or understand it,
will change either a frequency or an amplitude or both.

That is, by definition, what distortion does. And it rarely does this in a
linear manner. That is, some frequencies or amplitudes are changed more than
others. These changes in frequency and amplitude will change the pitch relations
in the music and thus alter the tune. For example: Since the perceived pitch of
a note consists of the sum of its fundamental plus its harmonics, a distortion
that adds extra harmonics will shift the pitch of that note up slightly.
Likewise, a distortion that results in the rolling-off of higher frequencies
(thus reducing the amplitude of some harmonics) can lower the perceived pitch.

Our musical scale is composed of a series of fixed steps. Each step is the
same size. Our brain has an uncanny ability to follow those steps and to
determine when errors have been made. It is much like climbing a set of stairs.
As long as all the steps are the same you can comfortably walk up the steps,
come down the steps, run up, run down, take two steps at a time, even do it in
the dark. However, change the size of just one step and you are likely to fall
on your face.

Following the tune is much the same. If you try to follow along with the tune
you will find that, on a good hi-fi system, the tune will seem to make more
sense. The steps will be more regular. The notes that one instrument is playing
will have some relationship to the notes that another instrument produces. You
will even frequently know what note is coming next. In the end, the better the
system the less damage it does to the pitch relationships and the easier it is
to follow the tune. And, since any type of distortion, regardless of its source,
must alter the tune, this method is a comprehensive test of a hi-fi system's
musical performance.

More Problems?

Some people, at first, have a little difficulty with this method of evaluation.
After all, even a transistor radio plays a tune you can follow. A key factor
here is the amount of work your brain must do to follow the tune. Your brain
can, and does, correct for errors in the tune. Your brain can figure out what
the tune was supposed to be. The worse the hi-fi, the harder your brain will
have to work to allow you to follow the tune. This is the concept behind the
dreaded "listening fatigue". If you should have any problems in detecting
changes in the tune, don't worry about it. Just sit back, relax, and try again.
You will eventually hear the difference. The remarkable thing is that once you
do hear the difference you will find that it is much more apparent than you
originally thought. The more you listen the easier this method will become.
Eventually you will find that you have a listening test that is consistent,
repeatable, and, best of all, a reliable indicator of the performance of any
hi-fi component.

A Final Word on A/B Demos

The approach we suggest when doing an A/B comparison is to listen to component
A, then listen to component B. If one sounds better, buy it. If they both sound
the same, buy the least expensive. Anything else is folly. We have always said,
"If it doesn't sound better, then it isn't better."

You will find it easier to compare components in an A/B situation if you play
only a brief passage (as little as ten seconds and certainly no more than thirty
seconds) on one component. Switch to the second component and play the same
passage. By keeping the passages short, you will have the "tune" fresh in your
mind and will be better able to judge the relative difficulties of following the
tune on each component.

Whenever possible avoid the use of comparators or switching boxes. The extra
connectors will degrade the signal of both components under test, frequently
bringing the level of performance of both components down to that of the switch
box, making any meaningful evaluation impossible. For similar reasons we
strongly suggest that all component evaluations be done in a demonstration room
that contains only one set of loudspeakers (not counting home theater).
Additional speakers, even though not being used, will vibrate in sympathy with
the original sound source. This added noise does make it more difficult to
evaluate components. Not only does it make it harder to judge the tune, it tends
to favor a component that is more aggressive sounding and unfairly penalizes a
good component by masking some of the low-level, detailed information that it is
capable of providing under normal conditions.