Saturday, May 30, 2009

Trained listeners with normal hearing are used at Harman International for all standard listening tests related to research and competitive benchmarking of consumer, professional and automotive audio products. This article explains why we use trained listeners, and describes a new computer-based software program developed for training and selecting Harman listeners.

Why Train Listeners?

There are many compelling reasons for training listeners. First, trained listeners produce more discriminating and reliable judgments of sound quality than untrained listeners [1]. This means that fewer listeners are needed to achieve the same statistical confidence, resulting in considerable cost savings. Second, trained listeners are taught to identify, classify and rate important sound quality attributes using precise, well-defined terms to explain their preferences for certain audio systems and products. Vague audiophile terms such as “chocolaty”, “silky” or “the bass lacks pace, rhythm or musicality” are NOT part of the trained listener's vocabulary since these descriptors are not easily interpreted by audio engineers who must use the feedback from the listening tests to improve the product. Third, the Harman training itself, so far, has produced no apparent bias when comparing the loudspeaker preferences of trained versus untrained listeners [1]. This allows us to safely extrapolate the preferences of trained listeners to those of the general untrained population of listeners (e.g. most consumers).

Harman's “How to Listen” Listener Training Program

Harman’s “How to Listen” is a new computer-based software application that helps Harman scientists efficiently train and select listeners used for psychoacoustic research and product evaluation. The self-administered program has 17 different training tasks that focus on four different attributes of sound quality: timbre (spectral effects), spatial attributes(localization and auditory imagery characteristics), dynamics, and nonlinear distortion artifacts. Each training task starts at a novice level, and gradually advances in difficulty based on the listeners’ performance. Constant feedback on the listener's responses is provided to improve their learning and performance. A presentation of the training software can be viewed in parts 1 and 2

Spectral Training Tasks

There are two different spectral training tasks. In the Band Identification training task, the listener compares a reference (Flat) and an equalized version of the music program (EQ), and must determine which frequency band is affected by the equalization (see slide 5 of part 2). The types of filters include peaks, dips, peak and dips, high/low shelving and low/high/bandpass filters. The task is aimed at teaching listeners to identify spectral distortions in precise, quantitative terms (filter type, frequency, Q and gain) that directly correspond to a frequency response measurement.

At the easiest skill level, there are only 2 frequency band choices, which are easily detected and classified. However, as the listener advances, the audio bandwidth is subdivided into multiple frequency bands making the audibility and identification of the affected frequency band more challenging.

The Spectral Plot training exercise takes this one step further. The listener compares different music selections equalized to simulate more complex frequency response shapes commonly found in measurements of loudspeakers in rooms and automotive spaces. The listener is given a choice of frequency curves which they must correctly match to the perceived spectral balances of the stimuli. This teaches listeners to graphically draw the perceived timbre of an audio component as a frequency response curve. Once trained, listeners become quite adept at drawing the perceived spectral balance of different loudspeakers, and these graphs closely correspond to their acoustical measurements [2], [3].

Sound Quality Attribute Tasks

The purpose of this task is to familiarize the listener with each of the four sound quality attributes (timbre, spatial, dynamics and nonlinear distortion) and their sub-attributes, and measure the listener's ability to reliably rate differences in the attribute's intensity. For example, in one task the listener must rank order the relative brightness/dullness of two or more stimuli based on the intensities of the brightness/dullness of the processed music selection. As the difficulty of the task increases, the listener must rate more stimuli that have incrementally smaller differences in intensity of the tested attribute. Listener performance is calculated using Spearman’s rank correlation coefficient which expresses the degree to which stimuli have been correctly rank ordered on the attribute scale.

Preference Training

In this task, the listener enters preference ratings for different music selections that have had one or more attributes (timbre, spatial, dynamics and nonlinear distortion) modified by incremental amounts.

By studying the interrelationship between the modification of these attributes and the preference ratings, Harman scientists can uncover how listeners weight different attributes when formulating their preferences. From this, the preference profile of a listener can be mapped based on the importance they place on certain sound quality attributes. The performance metric in the preference task is based on the F-statistic calculated from an ANOVA performed on the individual listeners’ data. The higher the F-statistic, the more discriminating and/or consistent the listeners’ ratings are --- a highly desirable trait in the selection of a listener.

Other Key Features

Harman’s “How to Listen” training software runs on both Windows and Mac OSX platforms, and includes a real-time DSP engine for manipulating the various sound quality attributes. Most common stereo and multichannel sound formats are supported. In “Practice Mode”, the user can easily add their own music selections.

All of the training results from the 100+ listeners located at Harman locations world-wide are stored on a centralized database server. A web-based front end will allow listeners to log in to monitor and compare their performances to those of other listeners currently in training. Of course, the identifies of the other listeners always remain confidential.

Conclusion

In summary, Harman’s “How to Listen” is a new computer-based, self-guided software program that teaches listeners how to identify, classify and rate the quality of recorded and reproduced sounds according to their timbral, spatial, dynamic and nonlinear distortion attributes. The training program gives constant performance feedback and analytics that allow the software to adapt to the ability of the listener. These performance metrics are used for selecting the most discriminating and reliable listeners used for research and subjective testing of Harman audio products.

Thanks, Sean. I have a little arts blog, GrumpyOldArts. The audio section seems to be generating the most interest, so I'm thinking of focusing on it a bit more. I have no interest in measuring gear and publishing specs, etc., but a lively discussion of great recordings and the stuff we play it on, without the usual audiophile voodoo, would be of great interest. Your software sounds like something that would focus and facilitate that discussion. I look forward to its availability.

This is exacly the kind of program I've been looking for. Ik sure hope you'll find a good way to make it available to the public. The already proposed web-based frontend would be a way of mutual benefit. I've used an ABX-comparator and files on the pcabx site, but its nothing like the program described above. Very much looking forward to it!

Hi Steve,Thanks for your comment. We hope to make the training software public before the end of this year. I agree that having a common descriptive language is not only a benefit to Harman listeners, but would be great tool for audio reviewers and consumers alike.

Hi Valentin,My AES paper on the Harman Reference Listening Room is currently only available on the 126th AES CD-ROM: http://www.aes.org/publications/conventions/AES does this so they sell more CD-ROMS, then they make it available as a single paper about 6 months after the convention. That means it should be available as a single paper by October.

Having a common lexicon for listening would be a boon of unprecedented proportion. First, it would demystify the process. Second, it would eliminate the voodoo power-mongering at the so-called elite levels. Third, it would level the field for brands like Infinity, JBL, and Revel, who compete with art and science against the magical incantations and secret society logic of some brands and their priests and acolytes. Fourth, it would empower enthusiasts and audioplhiles (yes I use that word in its literal meaning) to make informed decisions based on sound principles. Fifth and finally, it might restore some fun and order to the whole audio pursuit.

Hi DougNDO:Thanks for your comments. We are working on two things that we hope to make public: 1) The Harman "How to Listen" software that I talk about in this article and 2) an Audio Glossary that Floyd Toole is writing, and will include the subjective evaluation terms we use for listening tests.

You know, I have this set of Dave Moulton's Golden Ears training CD's that I'm beginning to lend to SR staff to help them self-train their own ears to improve their ability to do their work. If there has ever been any usefulness or market for that CD set, then I must assume there will be even more so for your software. I realize that isn't the focus or purpose, but that's exactly why I'm pointing this out: it can serve more purposes than you might have thought.

It can be especially handy for helping "technical volunteers" gain some critical hearing skills in such organizations as churches and community radio stations where these people are not otherwise "into" audio as much as the rest of us. Yet those are precisely the folks/organizations who would not have a lot of money to spend on such training. If you make this tool freely available, it can be a good thing for lots of folks. If you charge a license fee for it, consider a break for charitable orgs, please? :-)

I must say, Harman is doing really great things lately. The LSR lines are great, as is the Vertec. Oh how far you've come in 10 years! I'm rather excited to try this software, and utilize these terms. Thanks for what you are doing!!

I just finished Floyd Toole's book and have enjoyed reading each of your posts. As a consumer who is both untrained in listening and measurement, I am very curious to know if the listening software you write about is available as a self-tutorial? I am also very curious to find out how one might accuately measure the in-room response of their speakers - I can see that my spectrum analyzer is really not the ticket - but what do you recommend for people to optimize sound in their own room? Is SFM an off-the-shelf product?

I am a two-channel listener with hearing something like Dr. Toole has as outlined in his book. Sorry for the many questions...

Hi Nick,The "Harman How to Listen" listener training software is not yet available - but we hope to make it available soon. It is self-guided and doesn't require an instructor per se.

I just posted a new article in thie blog about loudspeaker/room correction: see http://seanolive.blogspot.com/2009/11/subjective-and-objective-evaluation-of.html

For room correction DIY's you need high resolution frequency measurements (1/20-octave resolution) with the ability to post-smooth the spatial averages. There are some low cost commercial measurement systems that can do this I think like ETF (PC) or Fuzz Measure (Mac). You need a calibrated 1/2-inch or 1/4-inch pressure microphone and stand that you can place at ear height on the listening seats(s). You need to do a spatial-average of the measurements since you don't want to equalize a peak or notch that is very location-specific. Then you need a 2-12 band parametric equalizer used to equalize the in-room response to a target function. If you read my recent blog posting and accompanying PDF presentation you will get a general idea of what the target curve should look like.

The software described here sounds excellent. Do you have an updated timeframe for when it might be released? And any news on whether there might be a binary (or even better, the source code) released for Linux?

Hi Smably,I think we are looking at around a March release for the training software. We haven't compiled or tested a Linux version but this could be easily done. At this point, there are no plans to release the source code.

will you publish the specifics of the room treatments used in the most recent room?i am surrounded with revel ultima II product, and room conditioning is an ongoing process. Your insights are greatly appreciated.walt

We are using 4 foot square (3-inch) thick fiberglass absorbers on the walls, some custom hemicylindrical diffusers and a few RPG Skylines to reduce flutter echoes. The rest is just furnishings and a removable 1.5 inch wool area rug with foam underlay. The diffusers were made using 2 ft diameter "Sono tube" cut in half with plywood on the back, lined with dynamat and filled with fibreglass.

I too am much interested in your training software - any increase of objectviness in audio is much appreciated. I design audio equipment as a hobby, and testing is a big part of that - I live in LA too. Is there any chance to get a tour of Harman's testing facilities?

The download link for the Mac version seems to work fine for me. Try this again:https://doc-10-18-docs.googleusercontent.com/docs/securesc/s696v3vk61e58p9j1o627vjuhvml4tgo/69gv122n6kddl0rnensa95d2rkvpk6p0/1327527900000/08175448946673264289/08175448946673264289/0B97zTRsdcJTfZTU3MGE3YjgtOWVmOS00ZTFjLThmNWItNjQ1YTZiMjY1ZTdj?e=download

Sean, Do you recommend wide dispersion speakers in normal asymmetric living rooms with a multitude of reflective objects of all shapes, sizes and distances and without any room treatment of any type? (Married)

Seth,Good question: In an asymmetrical room with wide dispersion loudspeakers you might get some lateral sound stage bias towards the stronger/earlier sidewall reflection, which could be ameliorated by absorbing the first side reflection. But generally, the first arrivals will dominate the localization of the auditory images with the reflections only adding spaciousness/image widening.

Recommended Books

Subscribe To

About Me

Sean Olive is Acoustic Research Fellow for Harman International, a major manufacturer of audio products for consumer, professional and automotive spaces. He directs the Corporate R&D group, and oversees the subjective evaluation of new audio products including Harman's OEM automotive audio systems. Prior to 1993, he was a research scientist at the National Research Council of Canada where his research focused on the perception and measurement of loudspeakers, listening rooms, and microphones. Sean received a Bachelors degree in Music from the University of Toronto, and his Masters and Ph.D. degrees in Sound Recording from McGill University in Montreal. His Ph.D. research was on room acoustic adaptation and the acoustical interaction between loudspeakers and rooms. Dr. Olive has written over 30 research papers on the perception and measurement of audio for which he was awarded the Audio Engineering Society (AES) Fellowship Award in 1996, and two Publication Awards (1990 and 1995). Sean is the current President of the Audio Engineering Society. For more info see www.linkedin.com/in/seanolive