Sunday, January 20, 2008

Getting Your Fingers Caught in the Chinese Finger Trap of Traditional Development

One of the difficulties in moving to Agile development is that much of the “knowledge” that we have from decades of traditional development makes Agile seem counterintuitive. Part of this knowledge is embodied in habits, taboos, and ceremonies. There are things which we believe are facts in general which are actually only facts within the framework of traditional development. In effect, Agile is right in our blind spot.

Many of the practices of traditional development are actually accommodations for problems that only occur in a traditional project and thus are not necessary in an Agile project. We feel that we need these accommodations because they have always been there, but we need to recognize them for what they are in order to let go of them and embrace Agile development.

In traditional development, a frequent solution to many problems is adding more rigor, taking more time, adding more detail, and adding more policies and procedures. “This time we’ll do things right" is our frequent refrain. We believe the reason for problems last time was a lack of discipline and so there is a call for more discipline. This is analogous to the Chinese Finger Trap. The solution appears to be adding more policies and procedures (pulling harder on the finger trap), but in fact the solution is to simplify (pushing your fingers together.)

Tighter Policies and ProceduresIn any development project, problems arise. They may be new problems or old problems. At some point, it will be decided to make a change in response to a problem. It may be that the change is to add a new policy or a new procedure. Whatever the change is, it will likely require additional effort. The effort is justified by the fact that the problem is addressed. It may be addressed by treating the symptoms or it may be addressed by removing the root cause of the problem. When it does in fact require additional effort and does not remove the root cause of the problem, you have now increased the overhead of all future projects. This overhead contributes to the complexity of future projects and can lead to additional problems and additional overhead.

The impact of any change to policy or procedure is difficult to gauge in a traditional project. There just aren’t enough opportunities for feedback and adjustment. In an Agile project, the practices of short iterations and one piece flow provide constant feedback and frequent opportunities for adjustment.

Tighter RequirementsHow many times have you heard or thought something similar to: “The last time we didn’t get the requirements right, we better spend more time getting it right and get more details.” And what about the follow-on which probably isn’t said aloud: “Let’s make sure it is clear to everybody that we did the right thing and that nobody can point the finger at us and say that we didn’t do our job. Let’s make the requirements look as professional and complete as possible.”

Over time, these two sentiments produce a lot of boilerplate and CYA-style artifacts which only provide value after the fact when the finger-pointing starts. You can refer to the requirements and say “but see, we did what was asked of us.” So the value that is produced is that you can absolve yourself of blame. But nobody will pay you for that absolution and in fact you may lose opportunities because you didn’t satisfy the customer.

There are only four things which provide value in a software product: getting enough information from the customer to produce something which can then be used for further refinement, documenting how the software works just enough so that somebody can maintain it later and join the project now, providing software to the end user that they want, and getting paid for the software you produce.

Agile projects do use requirements, but only the absolute minimum required: no more and no less. The use of short iterations means that you will find and implement your customers' true requirements much faster so there is much less need for speculation and prediction. Requirements that turn out to be for things that customers don't actually need can be abandoned instead of elaborately detailed prior to the first customer feedback on working software.

For those that have auditing requirements, consider that if you have enough documentation for somebody to maintain your software then you should have an auditable “paper trail.” Conversely, if you don’t have an auditable paper trail, how can you expect to maintain your software or easily add a new team member?

There are many other areas of traditional development which contain Chinese Finger Traps.