Build One to Throw Away

Search

This is a maxim from Fred Brooks’
The Mythical
Man-Month. These days I’m thinking it’s the single most
important lesson there is about software.
It’s been brought rudely home
to me by my recent work on
mod_atom,
whose design is terribly simple; but I still got the first cut wrong in
important ways.

What I Got Wrong ·
mod_atom is about as simple as an AtomPub server can be; all the entries
and feeds and collections are backed one-for-one by ordinary flat files,
with entries scattered across a YYYY/MM/DD tree, just like here at
ongoing.

I had two URI/filesystem hierarchies, rooted at
atom/ and pub/, to separate the resources that exist
for AtomPub, like Service Documents and Media Link Entries, from those that
are there to be read, like feeds and HTML pages; the goal was to make access
control easy.
How could that be wrong?

What happened was, I went back, after a few months of ignoring mod_atom, to
build in
Meta-pubs, so you can do
CRUD on whole publications, not just
individual entries and media files. This ended up touching most pieces of the
system, and I realized that a couple of the core ideas were leading me into
all sorts of tangled-code complexity; in particular the double directory structure.

So I rewrote it with one filesystem tree routed at
pub/, backing two URI spaces which were identical except for
being rooted at pub/ and atom/.
The mod_atom code forces all PUT and POST and DELETE requests to go through
the atom/ subtree. This allowed me
to slash literally hundreds of lines of code.

There was another design error, just as obvious in retrospect; its cleanup
also allowed subtracting severe gnarl and great big chunks of code.

Programmers experience soaring joy when they can rip
through code deleting functions and declarations, screens-full into
the bit bucket, with the steady drumbeat of tests-fail-then-pass.

So maybe I didn’t build one to throw away, but I built one that needed
major amputations out of the box.

And Your Point Is? ·
This is by most standards a simple system; at the
moment, excluding various XML- and HTML-munging libraries, less than six
thousand lines of code. I’m an experienced programmer, have written hundreds
of thousands of lines of C code, know the internals of httpd pretty well, and
understand AtomPub as well as anyone in the world.

I’m not a truly great developer (I know this is true because I’ve worked
with some) but I’m not a moron either (worked with some of them
too) so if I’m going to make this kind of misstep, a whole lot of other people
will too.

And let me make a strong statement: I’m not sure there’s anyone living,
including your five favorite programming heroes, who could tell you,
without writing the code,
whether one or two directory trees were going to work out better for this
problem.

Given that, what hope is there for waterfall
development? Or for any approach that doesn’t leave space for going back and
building things right once you’ve learned what “right” is by building things
once? Well, none.

But You Already Knew That ·
Probably true. But you know what, how about we all agree, all of us who
write about writing software, to write about this once every year or so.
Because there’s this
terrible glaring conflict between what sensible managers want and what
sensible programmers know.
Managers, good managers, want a plan; they want to lock in design constraints
so that work can be dealt out and progress tracked and promises
kept. Programmers, good programmers, know that they’re not smart
enough to get the core design choices right until they’ve built something
that works.

The various techniques and disciplines gathered around the banner of
“agile” are on balance more honest at facing up to this unavoidable tension.
But there’s still lots more work to be done.

And the most important thing is, we all have to remind ourselves, all the
time, that we’re not smart enough to get anything important right the first
time.

Exceptions? ·
Yes, of course. If you’ve just written three driver-scheduling systems or
foreign-exchange systems in a row, you’ll probably go into the fourth with a
pretty good grasp of what’s important. And those kinds of systems matter.
But the interesting software is by definition the stuff that isn’t
the fourth iteration of anything. Perhaps it’s best to close with Brooks’
quote in full:

“Where a new system concept or new technology is used, one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time. Hence plan to throw one away; you will, anyhow.”

The next step is to look for the trigger that enables the second generation. Adding a feature in this case? Or Just the need to crawl all over the code?

If you're doing this for someone else btw, ensure you 'lose the source code' of the first iteration which then allows you to do the better job the next time round. If necessary lose the source code, you retain the ideas that worked the first time and might find the insight necessary to make the step improvements Tim talks about. The killer is taking your 'demo' system through to production use when you designed it as a knock up demonstrator! Yuk.

There is a big difference between rewriting parts of the code (even the core parts) and starting from scratch (throwing old code away).

It might sometimes look easier to start from scratch (especially if its not your codebase) but that means rewriting and retesting boring parts (boilerplate and glue code) as well. And in most codebases I have seen this boring parts vastly outnumber the core (interesting) parts.

So rewriting yes, but for throwing away you better have a really really good reason (ie. it was written in cobol and you can't find programmers to maintain it anymore)

This is the kind of article that needs to be written for CIO or some other managerial magazine, since as you said, us developers already know these lessons well.

A lot of managers I've experienced (yourself excluded of course, Tim) look at software projects in a very linear fashion. Always going forward, and when a feature is complete you close the book on that chapter and don't look back.

Let's a strong lack of desire for a "regrouping" phase where you pause and review your experiences to that point. Sadly this typically leads to code that becomes more and more spaghetti as time goes on.

I agree, the top bloggers should write such an article once a year to keep these important lessons in focus.

<em>"Build One to Throw Away - This is a maxim from Fred Brooks' The Mythical Man-Month. These days I'm thinking it's the single most important lesson there is about software."</em>

That's rather peculiar, as Brooks himself has pretty much retracted that statement: The current edition contains a chapter <em>TMMM after 20 years</em>, which reads: "'Plan to throw one away; you will, anyhow.' This I now perceive to be wrong, not because it is too radical, but because it is too simplistic." (ISBN 0-201-83595-9, p. 265)

Well-told from the programmer's view. But imagine if you are an executive.

Say you can spend $5 million on R&D projects. Each of the 50 projects proposals you look through says "I don't know how much it will cost, or how long it will take until I get done. But that's ok, because Tim Bray says so in his blog."

What projects do you fund? Can you fund 5 or 15 projects? How do you get an idea of what your return will be on what you invest?

Would you hire a contractor to paint your house, if he says "I'll tell you what it will cost, and how long it will take when I'm done."