Incremental Development

Most modern methodologies talk about incremental and iterative
processes, but surprisingly few projects operate this way. Why is this? Everybody
is talking about it but nobody doing it?

As I have discussed elsewhere, before we can do incremental development
we need to do incremental
requirements capture. A more fundamental issue however is why do incremental
development in the first place? What is so important about incremental development?

The key reason as I see it is that there is always more software to be developed
than there are developers available to work on it. The obvious way to deal with
this is to prioritize the work. By defining different versions or releases of
feature sets we make it feasible for a team of developers to release something
useful. Rather than having to wait forever for the all singing and dancing version,
a minimalist version can be released quickly and then evolved to meet
the real needs of the users

Once the development team has managed to release something, the people who use
the application on a daily basis are in an ideal position to guide the evolution
of the application. The real skill in supporting a live application is the ability
to rapidly respond to user requests. After all if a user reports a problem it
doesn’t really work to say "that will be fixed in the next release"
when the next release is six months away. What the user really wants is an instant
fix, or failing that a fix that comes fast enough so that their work is not
adversely affected.

Admittedly shrink-wrap software vendors currently get away with delayed fixes,
but for the bulk of application software that is developed by small teams, delays
are not an option. When users are relying on your application to get their work
done you either provide a fix quickly or the users find someone else who can
guarantee a quick fix.

Users are more patient with requested changes as opposed to fixes, but even
then a six month wait is not desirable. Users have a feel for the size of the
change that they are requesting, so are unwilling to wait for small, high priority
changes.

This is why the concept of Capturing Requirements for Prioritization
is so important. All requests need to get fed into the same Work Queue
so the users can see how high priority items bump the other, lower priority
items to the back of the queue.

Release frequency depends on the priority of the items in the work queue.
Show-stoppers are not gong to be batched up, they will go live as soon as they
are ready, change requests will be batched up and released when it is convenient
for the users.

The interesting part about a live application is that it forces developers to
work in an incremental process. If the team ever try to make a big, monolithic
change a high priority bug will force the developers to create a special bugfix
version of the source. Maintaining the application then becomes a nightmare
as all of the bugfixes have to be merged back into the mainline source tree.

Instead what the developers have to do is work incrementally,
always making sure that application as a whole is in a releasable state. To
do this developers have to make small, incremental changes to the application,
such that the overall application is never broken. They have to pick the top
priority item off the top of the work queue and work on it as a team to integrate
the new and changed functionality into the application. Only when it is completely
done is it safe to move onto the next item in the work queue.

This discipline of only working on one thing at a time is essential because
the users are not very forgiving when it comes to slow delivery of high priority
items. Developers have to learn to gang up on requirements and change requests
to minimize the time taken to implement the features. When it comes to developing
high priority items, efficient use of developer resources is not the driving
goal, shortening the elapsed time from request to release is the driving goal.

Since this style of working is so important once the application has gone live
it is important that developers get good at it before the application goes live.
This means that developers have to use incremental development before the initial
release to the users. After all there is hard to work in one style for a year
and then suddenly switch to a completely different style when the application
goes live.

By always letting the Work Queue drive an incremental development process
your team will get practiced at what is really important; learning to
become a responsive software development team.

Interestingly what I have been calling the Work Queue is what used to be called
the "application backlog". The requested items that
the developers hadn’t got around to working on because there were other
higher priority items to be worked on. My contention is that using incremental
development to make the development team responsive to high priority requests
completely changes the conversation about the outstanding work.

If the users have to wait forever for even minor changes then we have an unresponsive
team and an application backlog. With a responsive team the users know that
whatever they choose to put at the top of the work queue gets done really quickly,
and for these users there is no application backlog. There is
just a queue of low priority items that will be ready when they are really needed.