Friday, 1 February 2008

Prototypes and Real Applications

There is an essay, The Art of Lisp &
Writing by Richard
Gabriel. It's long, but like all of Gabriel's
essays it is worth
reading if you're interested in exploring alternative views of the act
of writing programs.

I am simply not qualified to do justice to Gabriel's central
point. Simply note, it was Brooks of The Mythical Man Month, that
bible of solid, empirical, meticulous software engineering planning
who first drew the connection between the work of programmers and
poets.

Instead, there is a minor point in Gabriel's essay that caused me to
think about a brief chat in a recent interview. We'd worked our way
through the questions, the candidate had done well, there was enough
time left to just chat about his work. His two favourite languages
were C++ and Python, so we drilled a little there. How had he used
them? What did he think of them? Had he tried to combine them
together?

Well, it turned out he had. Using boost::python he had exposed some
of his C++ classes into Python. This was interesting. Why? It was so
he could quickly produce parts of the application; he'd become a fan
of this technique. All very well and good, had he tried that approach
elsewhere? Unfortunately, no he hadn't. His other use for Python was
as a prototyping language. So close. I drilled a bit on this
point. Had he made the obvious step from here? Turns out he
hadn't. But that did leave me thinking, and Gabriel's essay
crystallised this issue in my mind.

Why did he just throw away his Python prototype and rewrite the whole
thing in C++? Why didn't he grab a profiler and start replacing the
slow Python parts with fast, small chunks of C++?

Developers seem to see an enormous gulf between a prototype and a
'real' application. Something that is appropriate for a prototype must
be discarded for the real version. The basis for this act seems to be
that a prototype is a quick and dirty hack and can't be trusted to
work.

Well, I don't know about you, but I can't help myself: every piece of
code I write, I write maintainably and flexibly. And these are issues
in prototypes. A prototype is meant to be an exploration, a journey to
see how an idea will play out once it's embodied in a program. It is
an idea that has not been seen before - this is when it needs to be
most flexible. When everyone who sees is thinks of an improvement,
when the sight of running program inspires completely new ideas. This
happens during the prototyping phase - and if you don't explore these
ideas, then what was the point of the prototype at all?

Your prototype needs to be written quickly and then it needs to change
quickly. You'll only be able to do that with a maintainable, flexible
code base. In short, a well-written code base. You're a proficient
software engineer, you know how to do this. You probably do it without
even thinking.

And at some level, everyone knows this. That's why prototypes are
created in languages like Python. A language that you can write
quickly, but also write well, quickly.

So when it comes time to write the 'real' application, when all the
decisions have been made and the exploration has stopped, when we know
what the program will do, why do we throw away all that very carefully
engineered prototype code? It can't be because the code is no good,
because that code is better than you believe the 'real' program needs
to be.

Most developers would say that it has to be rewritten for
performance. But when was the last time you profiled your code?
Because if you haven't profiled you simply can't know what your
performance is, and more importantly, where the trouble-spots are.

I want to prototype. I want to explore my ideas in a running
program. I want to carefully engineer my program and then change it at
a moment's notice. And I want to be able to do that all the time, for
the entire life of the program. I don't want to kill my program just
when it's about to finally get some users.

I want to express my ideas in a language that gives me all that.

Oh, and that application the candidate had been working on? The one
that had Python embedded in the real, final version? It was a
massively multiplayer 3D game engine.

5 comments:

He knew python, but it wasn't an in-house official language. His boss was lenient in letting him choose the prototyping language, but after version 1, he couldn't get help from teammates. The benefits of working in a C++ team outweigh the benefits of working alone in python.

At least one of the places where he was prototyping with Python also used Python as an official language. As far as I could tell it just never occurred to him to treat the prototype as a solid piece of work.

I think you are confusing "Prototype" with the "Tracer Bullet" concept. Check out the book: "The Pragmatic Programmer".

I do understand the frustrations with people treating a perfectly good language as some kind of bastard child. My company has an application written in Delphi. Despite its success, management would like to get rid of it.

Wow, I didn't know anyone was stupid enough to ignore Burn The Disk Packs. For the record, burning your prototype is EXACTLY what everyone should do. If anything, in this case the guy probably didn't burn it completely.