Classic Testing Mistakes – A Look Back

Back in 1997 Brian Marick (who later became one of the 17 original authors of the Agile Manifesto) wrote an article called “Classic Testing Mistakes” that had become fairly well known in the so-called QA community at the time. According to Brian’s website, this is the paper that caused James Bach to exlaim, “Brian! I never thought you know anything about testing before!” at that time. It’s interesting to take a historical look at this paper, and see what has changedover the past almost 19 years. There is a lot to examine here, but I’d like to highlight two brief ideas:

Marick starts with the role of testing. “A first major mistake people make is thinking that the testing team is responsible for assuring quality”. I think that by now, many organizations have abandoned that antiquated idea. Quality is the responsibility of the entire organization, we are now told. Indeed, I can’t think of a single place where I’ve worked where “Quality” was not listed on the company’s list of Core Principles. On the other hand, there may be certain organizations in certain contexts where the “tester-gatekeeper” model might still sometimes be valid. Once again, context is the key.

Along the same lines, Marick continues with a second classic mistake:”Most organizations believe that the purpose of testing is to find bugs” Instead, he explains, there is one key word missing: Testers should be finding important bugs. “Too many bug reports from testers are minor or irrelevant, and too many important bugs are missed”, he writes. This is an interesting distinction, because it appears to be an early formulation of today’s context-driven working definition of quality and testing. The now-famous definition of quality by Jerry Weinberg, “Quality is value to some person,” implies that testing is inherently subjective. What may be considered a bug to one stakeholder might be a key feature to another, therefore, the first order of business for the software tester is to understand what is important to their users so that they can focus on finding the important bugs, if that’s what they need to do.

Please take a look at Marick’s paper. Which of these do you think still apply? Have any of them changed? How?