Back to Waterfall Isn't QUITE the Right Answer

An odd article by Trent Nix on "LearnTFS" purports to debunk Agile by moving closer to waterfall. Interesting, but not quite right.

Trent Nix, in this article, suggests that you can make an Agilist “squirm” by asking where “your QA process” fits into Agile. After the obligatory snide commentary about the “Agile miracle”, the article addresses a common issue that arises when you’re doing Agile, and proposes a common inadequate solution. Here’s my reply:

The author has identified a real issue. So far, so good. Many teams trying Agile do encounter the test crunch/chaos at the end of the iteration. Some do go to a follow-on testing phase in the next iteration.

Bug reports later are not better than bug reports sooner.

It turns out that doing the testing later is not a very good “solution” to the concern. The reason is that while developers are working on D, E, and F, the testers are finding defects in A, B, and C. These defects come bounding back in, fouling up the current iteration and the next ones.

The author’s curative suggestion is that you should “subtract time from your iterations for bug fixes”. Certainly, if you’re fixing defects, it will take time. The problem is that this time is not able to be planned. Yesterday’s weather doesn’t work.

Time lost fixing defects is still lost, even if you account for it.

More important, fixing defects is rework. You are doing work over. This means that your estimates of how long it took to do the thing were wrong, often very substantially wrong.

Rework breaks Agile.

The result of this is that you lose the most important benefit of Agile, which is the ability to predict future delivery based on the completion of “done” software. It is the flow of “done” software that enables management by scope rather than by pressure.

Move testing upstream, not downstream.

Therefore, moving back toward waterfall is not the best next step, even if, as the author suggests, you shorten your iteration. A better thing is for developers and tester to learn to work together to get all the planned features fully done in a single iteration. This is neither simplistic nor easy. It requires great skill to determine as many checks for the software before the iteration starts, and to automate enough of these to let the developers run them as they develop, not just at the very end.

Examples clarify requirements and reduce defects.

When developers have well-crafted examples before they start, especially when they are available in some simple automated tool, they are smart enough to run them all before passing the software off to anyone else. If the software is then passed to testers, those testers can run the automated checks quickly if they care to, but are able to spend their valuable time looking for something interesting, rather than manually checking basic results.

Desire for stabilization iterations tells us we're not good enough yet.

Another idea that our author supports is the notion of having one or more “stabilization” iterations. If these are needed, they are evidence that the work was not done at the time it was declared done. This violates the fundamental principle of Agile. Stabilization iterations are evidence of trouble, not a good idea.

Integration and regression testing are almost always needed. The construction of these automated checks is very valuable in making this testing smooth, rapid, and repeatable. Quite commonly they can remove or at least reduce the need for “stabilization iterations”, another idea that Trent supports.

Strong testing skills are very important to the team.
Rote testing, not so much.

All of this work is best done with good testing skills on the team. The trick is not to move the testers downstream: the author got that exactly wrong. The trick is to move them upstream, where they help the Product Owner describe what is needed in terms of clear automated examples. Their skills in identifying details and edge cases then inform the team the first time they work on the story, not just when they work on the bug reports.

Good teams do this. So can you.

The best Agile teams can produce software that is essentially defect-free in a single iteration. They can keep the whole system integrated all the time, fully tested all the time. Trent Nix has identified a common problem. Unfortunately, he has not proposed the best known solutions.

Good questions are a good start. It’s worth noting that people have been very successful doing Agile for about fifteen years now, and that there are good answers out there. So don’t just stop at the questions and then drop back into your old ways.