A pilot study was conducted in order to investigate the application of descriptive analysis techniques to the evaluation of consumer audio equipment. A global seven-step methodology for the descriptive evaluation of the sound characteristics of consumer audio equipment was developed. In contrast to the commonly used forced choice discrimination methods for audio equipment evaluation, the proposed methodology enhances, rather than compromises, a subject's sensory input. A case study employing a minimally trained subject demonstrated the applicability of sensory evaluation techniques to sound. There has been some resistance in the audio community to the adoption of sensory science evaluation methods because such methods are erroneously thought to apply only to products that affect the senses of touch, taste, smell and sight.

discuss.

btw, Dr. Smith is also the author of this rather curious article in 'Affordable$$Audio''

It appears that his main justification for rejecting ABX testing is that it was once used to test audio quality of telephones.

He says:

"Blind audio testing, which includes visually obscuring all or part of the sound stage, rapid switching of musical selections and off-axis and groupseating, impairs the listener's ability to localize sounds (seeing), to internalize and evaluate aural cues (hearing) and toreceive correct stereophonic tactile information (touching). Any stereophonic audio system testing methodology whichcompromises and hinders the processes of human sensory perception will result in consistently inaccurate and oftenabsurd results."

This is one of the most confused paragraphs I've ever read about listening tests. It is a total misrepresentation of the true facts.

Since when does ABX test include "visually obscuring all or part of the sound stage"? I guess the author thinks that blind testing necessarily means wearing blindfolds, poking one eyes out, or darkering the room.

As has been pointed out many times, rapid switching of musical selections is not part of ABX testing. What may be rapdily switched is which presentation of the musical lsitening selections is being listened to at any particular time. Or not.

Since ABX tests are frequently done with just one listener, and since headphone listening is commonly done during ABX tests, the comments about off-axix and group seating are equally irreleevant to the true factys.

This paper is bascially a straw man argument. The apparently athor makes up his own ideas about what a bad listening test might be like, and then names them "ABX". The artile looks to me like baldfaced libel. I'd be worried if the author had any influence.

It appears that his main justification for rejecting ABX testing is that it was once used to test audio quality of telephones.

He says:

"Blind audio testing, which includes visually obscuring all or part of the sound stage, rapid switching of musical selections and off-axis and groupseating, impairs the listener's ability to localize sounds (seeing), to internalize and evaluate aural cues (hearing) and toreceive correct stereophonic tactile information (touching). Any stereophonic audio system testing methodology whichcompromises and hinders the processes of human sensory perception will result in consistently inaccurate and oftenabsurd results."

This is one of the most confused paragraphs I've ever read about listening tests. It is a total misrepresentation of the true facts.

Since when does ABX test include "visually obscuring all or part of the sound stage"? I guess the author thinks that blind testing necessarily means wearing blindfolds, poking one eyes out, or darkering the room.

As has been pointed out many times, rapid switching of musical selections is not part of ABX testing. What may be rapdily switched is which presentation of the musical lsitening selections is being listened to at any particular time. Or not.

Since ABX tests are frequently done with just one listener, and since headphone listening is commonly done during ABX tests, the comments about off-axix and group seating are equally irreleevant to the true factys.

This paper is bascially a straw man argument. The apparently athor makes up his own ideas about what a bad listening test might be like, and then names them "ABX". The artile looks to me like baldfaced libel. I'd be worried if the author had any influence.

That is a big a$$ compound sentence but it is actually true if you break it down for what it is and only what it is. He is describing a specific subset of "blind audio testing." That subset includes a visual obscuring of the "sound stage" (not sure why this would be an issue) rapid switching of musical selctions, and "off-axis and group seating." The last part is an issue. You will mess up the imaging of stereo playback *if* you do test that way.

If the author makes any broad inferences about blind testing from this specific subset conditions then that would be problematic to say the least.

That is a big a$$ compound sentence but it is actually true ihttp://www.hydrogenaudio.org/forums/index.php?act=Post&CODE=02&f=40&t=83979&qpid=725107f you break it down for what it is and only what it is. He is describing a specific subset of "blind audio testing." That subset includes a visual obscuring of the "sound stage" (not sure why this would be an issue) rapid switching of musical selctions, and "off-axis and group seating." The last part is an issue. You will mess up the imaging of stereo playback *if* you do test that way.

If the author makes any broad inferences about blind testing from this specific subset conditions then that would be problematic to say the least.

The paragraph quoted is the *concluding paragraph* of the article. If he was writing objectively, Dr. Smith would have written something like, 'blind tests have been conducted where in addition to being blind, ....", clearly separating the conditions which are intrinsic to a blind test, from those that are not. As written, he appears to be trying to tar audio blind testing itself or 'at the very least' obfuscating the issue by the use of the ambiguous term 'include'. This is on top of multiple risible claims elsewhere in the article, like "According to Lipshitz, Vanderkooy and Clark, It is better to blindfold or otherwise visually handicap music listeners".

And btw, your reflexive need to undermine anything Arny writes remains as pathetic as it is predictable. Please be aware that some of us here know of the long history between you two.

That is a big a$$ compound sentence but it is actually true ihttp://www.hydrogenaudio.org/forums/index.php?act=Post&CODE=02&f=40&t=83979&qpid=725107f you break it down for what it is and only what it is. He is describing a specific subset of "blind audio testing." That subset includes a visual obscuring of the "sound stage" (not sure why this would be an issue) rapid switching of musical selctions, and "off-axis and group seating." The last part is an issue. You will mess up the imaging of stereo playback *if* you do test that way.

If the author makes any broad inferences about blind testing from this specific subset conditions then that would be problematic to say the least.

The paragraph quoted is the *concluding paragraph* of the article. If he was writing objectively, Dr. Smith would have written something like, 'blind tests have been conducted where in addition to being blind, ....", clearly separating the conditions which are intrinsic to a blind test, from those that are not. As written, he appears to be trying to tar audio blind testing itself or 'at the very least' obfuscating the issue by the use of the ambiguous term 'include'. This is on top of multiple risible claims elsewhere in the article, like "According to Lipshitz, Vanderkooy and Clark, It is better to blindfold or otherwise visually handicap music listeners".

And btw, your reflexive need to undermine anything Arny writes remains as pathetic as it is predictable. Please be aware that some of us here know of the long history between you two.

Yes Steve, "to say the least." Do you understand what that means? It was a punch line, a sarcastic insult in the form of an understatement. Arny quoted the specific sentence and felt it was " one of the most confused paragraphs I've ever read about listening tests. It is a total misrepresentation of the true facts." So my comments were limited to that paragraph. Well, actually it was just a sentence. I don't see my comments as "undermining" Arny. I was trying to have some fun with a poorly written sentence that was painfully convoluted. I doubt that Arny would disagree that blind testing in stereo with "off axis and group seating" is a problem. Obviously the conditions listed in that "confused" sentence/paragraph were so specific and pointed as to have little or no meaning in the big picture. Oh yeah, I kinda said that didn't I?

Dude, lighten up. I was making fun of the sentence/paragraph. Oh well, once one has to explain the joke the joke dies.

It appears that his main justification for rejecting ABX testing is that it was once used to test audio quality of telephones.

He says:

"Blind audio testing, which includes visually obscuring all or part of the sound stage, rapid switching of musical selections and off-axis and groupseating, impairs the listener's ability to localize sounds (seeing), to internalize and evaluate aural cues (hearing) and toreceive correct stereophonic tactile information (touching). Any stereophonic audio system testing methodology whichcompromises and hinders the processes of human sensory perception will result in consistently inaccurate and oftenabsurd results."

This is one of the most confused paragraphs I've ever read about listening tests. It is a total misrepresentation of the true facts.

Since when does ABX test include "visually obscuring all or part of the sound stage"? I guess the author thinks that blind testing necessarily means wearing blindfolds, poking one eyes out, or darkering the room.

As has been pointed out many times, rapid switching of musical selections is not part of ABX testing. What may be rapdily switched is which presentation of the musical lsitening selections is being listened to at any particular time. Or not.

Since ABX tests are frequently done with just one listener, and since headphone listening is commonly done during ABX tests, the comments about off-axix and group seating are equally irreleevant to the true factys.

This paper is bascially a straw man argument. The apparently athor makes up his own ideas about what a bad listening test might be like, and then names them "ABX". The artile looks to me like baldfaced libel. I'd be worried if the author had any influence.

That is a big a$$ compound sentence but it is actually true if you break it down for what it is and only what it is. He is describing a specific subset of "blind audio testing." That subset includes a visual obscuring of the "sound stage" (not sure why this would be an issue) rapid switching of musical selctions, and "off-axis and group seating." The last part is an issue. You will mess up the imaging of stereo playback *if* you do test that way.

If the author makes any broad inferences about blind testing from this specific subset conditions then that would be problematic to say the least.

I see nothing in the entire article that suggests that the author says hs is adressing just a subset of blind listening tests. If you can find it, quote it. If you can't, we'll chalk it up as just another one of your many hihgly flawed attempts to undermine listening tests that fail to justify your extensive and expensive purchases of non-functional and poorly-functioning boutique audio gear.

Dude, lighten up. I was making fun of the sentence/paragraph. Oh well, once one has to explain the joke the joke dies.

The dominant impression was one of contentious spirit and prevarication for me. You might add a bold-faced "Bazinga!" next time, so that the simple-minded among us also get the humor.

Thanks for the tip. Did not know that Bold face was internet code for dry sarcasm or something like that. Sorry for any misunderstandings. It was meant as a back handed compliment to the author of the article. Kind of like the old Monty Python bit, "what have the Romans ever done for us. besides the...blah blah blah...." only in reverse. He's right about DBTs if....blah blah blah.... The absurdity of the conditions being the irony. Well anyway...like I said... once ya gotta explain the joke the funny dies.

Dr. Smith's teaching and research interests are in the areas of Communications Signals and Systems, Broadband Telecommunications Network Design and Optimization, and Stochastic Modeling. Dr. Smith's consulting interests are in enterprise network and public switched network modeling, design and optimization.

The fundamental flaw in applying ABX tests to audio evaluation is in that people participating in ABX tests are searching for differences between equipment rather than conducting a performance evaluation against standard performance criteria (metrics).

In other words, ABX is fundamentally flawed because it compares gear to gear rather than comparing gear to performance standards. This is ludicrous and demonstrates an ignorance of the purpose of stereophonic music reproduction. This seems to be a concept that the "objectivists" and "scientists" cannot, or will not, grasp

.

It's saddening to read because he employs academic sensory testing jargon, and mixes 'real' science with tendentious crap. E.g., this amazing jumble of fact and fiction:

1. An ABX (duo-trio balanced reference) test generally requires a subject population of at least 16 persons. Optimum subject population is at least 32 or more persons.

2. An ABX (duo-trio balanced reference) test which employs a subject size of less than 28 persons generates high rates of beta error (false negatives or "no differences between samples") in the results.

3. An ABX (duo-trio balanced reference) test must compare samples which are unknown (unfamiliar) to the test subjects.

The fundamental flaw in applying ABX tests to audio evaluation is in that people participating in ABX tests are searching for differences between equipment rather than conducting a performance evaluation against standard performance criteria (metrics).

In other words, ABX is fundamentally flawed because it compares gear to gear rather than comparing gear to performance standards. This is ludicrous and demonstrates an ignorance of the purpose of stereophonic music reproduction. This seems to be a concept that the "objectivists" and "scientists" cannot, or will not, grasp

.

It's saddening to read because he employs academic sensory testing jargon, and mixes 'real' science with tendentious crap. E.g., this amazing jumble of fact and fiction:

1. An ABX (duo-trio balanced reference) test generally requires a subject population of at least 16 persons. Optimum subject population is at least 32 or more persons.

2. An ABX (duo-trio balanced reference) test which employs a subject size of less than 28 persons generates high rates of beta error (false negatives or "no differences between samples") in the results.

3. An ABX (duo-trio balanced reference) test must compare samples which are unknown (unfamiliar) to the test subjects.