The Testing Quadrants – We got it wrong!

The Testing Quadrants continue to be a source of confusion. I heard that Brian Marick was the first to write them down after long conversations with Cem Kaner. Lisa Crispin and Janet Gregory refined them in their book on Agile Testing.

I wrote earlier about my experiences with the testing quadrants in training situations a while back. One pattern that keeps on re-occurring when I run this particular exercise is that teams new to agile software development – especially the testers – don’t know which testing techniques to put in the quadrant with the business-facing tests that shall support the team. All there is, it seems, for them is critique to the product. This kept on confusing me, since I think us testers can bring great value. Recently I had an insight motivated by Elisabeth Hendrickson’s keynote at CAST 2012.

Two weeks ago, I attended CAST 2012. Elisabeth Hendrickson held a keynote there. If you haven’t already, you probably want to watch it:

In her talk Elisabeth provided a new interpretation of the testing quadrants that looks like a promising experiment for overcoming some of the drawbacks I had with the second quadrant, the business-facing tests that shall support the team. Elisabeth proposed a re-labeling of the two columns to confirmation vs. investigation.

With the new labels the quadrants are a bit different than the ones that Brian Marick originally wrote down. On the left-hand side the confirmatory tests that are technology-facing are of course the unit tests and those class-level integration tests that developers write. These help drive the development of the project forward.

In the second quadrant now are the expectations of the business. Usually I try to express most of them in the form of executable specifications, that I can automate. And I can also imagine other business-facing expectations that I cannot automate that probably fall in here, like reliability of the software product.

In the third quadrants on the top right are tests that help to investigate risks concerning the external quality of the product. For me Exploratory Tests fall in this category, but also usability concerns, and end-to-end functionality.

The fourth quadrant then probably consists of internal qualities for the code like changeability (static code analysis comes to mind), maintainability (how high is your CRAP metric these days?), and design flaws like code smells.

If you now wonder where quality attributes like performance and load testing are in this new model, I consider them to be either part of business-facing expectations, or they are part of the investigative process of external quality attributes. I think this depends on whether they have been made explicit early enough, or we find out during our investigations that load times are too low, for example.

I think there are more things to discover in this new model. Like with any models, they can help you think in a particular direction, but please don’t try to use this as your only guide. Use this model wisely, and apply critical thinking to your results.

P.S.: If you noticed the typo in the slide deck from Elisabeth you are not the first to recognize. If you watch the video, one of the attendees of the keynote points that out, too.