Search This Blog

Tuesday, June 12, 2012

Testing in many mainstream organizations is choked with low-value standardized documents that not only gobble up valuable thinking and testing time, but actively discourage thinking and impede good testing. While some testers can hope for relief from the document burden through the spread of Agile methods, this ridiculous situation isn't going away in a hurry. As a blog post by James Christierecently reminded us, the floodgates on “ISO/IEC 29119 Software Testing – the new international software testing standard” will soon open. I suppose it's possible that the new standard will wash away the documentation excesses we have now. I'm not holding my breath on that.

Test documents, whose sole purpose should be to serve the work, are instead driving and constraining the work. Absurdly, form is dictating substance. When, as in this case, the form is obese and bloated, it sucks up and squanders all the energy that ought to go into the real thing.

Some testers (many, I hope) refuse to be tyrannized by the supremacy of form. I want to reach into the mainstream where form dominates and help testers there to join us. I want to help them learn to think better and think for themselves. This blog post is a step in that ongoing effort.

After presenting my own webinar, I watched the recorded version of Alan Richardson'sexcellent webinar "Thinking Visually in Software Testing". The thinking tools and practices he describes there are so uncannily like my own that it prompted me in writing this post to think and write about my thinking. (I encourage you to watch Alan's webinar, if you haven't already done so.)

Form over Substance

When
I first began working for a consulting company, my project manager gave me a
mantra for billable work: “Never create anything that is not a deliverable to
the customer”.

She
was a brilliant PM who became an important mentor for me, but not all her
advice was equally stellar. That statement in particular put horrible shackles on my
work that I took years to shed completely.

The
problem was that my customer deliverables were formal, standardized prose
documents. For me to show value, I needed to end each day with sections and
sub-sections populated with tidy paragraphs, building up to the finished
product.

At
first, I was lucky. My company was only dimly aware of standards purporting to govern
test documentation, and so I created my own templates. But over time, my templates became our company standards. When I used them, I was so busy trying to adapt
the structures for each project that I could no longer work easily with them. Like
the templates for test documentation imposed in many companies, I found that the
structure—the form—acted as a constraint on the substance. The tail was
wagging the dog.

We Can't Test Without Thinking

Testing
is all about thinking. We think and rethink constantly. We think about how best
to gather information on our projects: what to read and look at, and who to
talk to and when. We think about how to approach the software and we develop test
ideas as we go. We create test models, often complementary models for testing
different aspects of the same piece of software. We plan and replan and then
plan again. We think about what we've discovered in testing and design what we're going to do next. If we're doing a good job, we don't ever stop thinking.

Prose
documents in preset patterns inhibit thinking and creativity. It’s useful sometimes to have a checklist of important things to
think about, but we can’t afford to let those checklists limit our thinking.Templates are not the best checklists.

Writing in sentences can
sometimes help me to simplify and think through a tangled knot of ideas. I
do occasionally write to understand what I’m thinking. But I
never set out to think in predetermined sections of formal standardized prose.
Do you? Does anyone? Can anyone?

Visual Thinking

A couple of years ago these drawings helped me think through a problem

Most
often, I think in scribbles and doodles, beginning with notes and drawings that
acquire structure as I develop the ideas and concepts. Or I might start with a
tentative visual structure to generate ideas and then modify or replace the structure
as needed to fit my thinking. Sometimes I scrawl ideas on coloured sticky notes
and move them around over several days on a board or double-page spread of my
notebook, drawing connections and annotating as I go. I often use mindmaps. I
may use several different techniques to get my hands around a difficult
problem.

Diagrams Emerge

What
comes out of my initial thinking processes is rarely a customer deliverable.
But over time, the result is usually some kind of structured diagram or set of diagrams that
I can then use to communicate my ideas to other people. Rather than dictating and constraining the substance, the form of these diagrams emerges from the substance.

Example of a strategy diagram for an end-to-end systems integration test on a large project

When—as is true on most
projects—I must produce a formal document, I prefer to put diagrams front and
centre. I want my documents to communicate, and I try to make them easy to read
and understand. I use prose as sparingly as I can get away with, using tables
and lists wherever possible. I don't include boilerplate, and I never copy wodges of text from one document to another. (Why
ever would I waste valuable project time on such useless make-work?)

This diagram shows the division of responsibility for testing on the same big project

Vacuous Form Tyrannizes the Mainstream

Apparently, that's not how most testers and test leads develop test documentation. In my consulting work with clients, I constantly see mammoth documents stuffed with books worth of low-content stodgy and opaque prose. Often, I search so-called test strategy documents in vain for any actual strategy. I fall asleep looking at test scripts that dictate every point and click and hideously repeat over and over again the most minute detail of so-called "test steps" and their piddling expected results. It's very hard to believe that much thinking is represented therein—or will inform the testing that must unfortunately follow.

Nobody reads this stuff. It isn't useful. It certainly doesn't help anyone test well. So why do testers waste time and spirit churning it out? Why do their managers insist on the tyranny of form over the substance that only thought can produce?

Let's Take Testing Back!

I believe that thinking testers must take testing back from the process weenies and form merchants. Many testers have done this, but it has yet to happen in the mainstream—in big companies, big banks, government projects...sometimes even in startups and small companies.

In subsequent posts on this topic, I'll explore some ways we can do this.