Paul Graham on Hacking

Paul Graham is a hacker, a
painter, and an essayist known as much for his thoughtful writings on spam,
hacking, and Lisp as for creating the Arc programming language. His new book,
Hackers & Painters,
contains many of his popular essays as well as several new ones. Paul
graciously agreed to an interview with the O'Reilly Network about his work.

O'Reilly Network: In the essay Hackers and Painters, you seem to argue that architecture is beautiful while engineering is more mechanical and less creative. Is this accurate, or am I putting words in your mouth?

Paul Graham: No, I wouldn't say that. I said that
architects decide what to do and engineers decide how to do it. But how can be
just as beautiful as what.

ORN: The C2 Wiki documents an anti-pattern called Architects Don't Code. Is
the distinction between architects and engineers useful in the world of
software? Returning to your "Hackers and Painters" essay, you said that, at Yahoo:

"Programmers were seen as technicians who translated the visions (if
that is the word) of product managers into code."

Or do you perhaps see hackers as architects who discover the software as
they hack?

PG: In theory you could design software without
implementing it, but in practice the best designs are rarely made that way.
The best programmers combine architect and engineer in one head.

ORN: Is this kind of design then, a process of
experimentation and experience? When you first face a new project, how do you
approach it?

PG: I try to build big programs out of small ones. So when
I approach a new project, I look for the subset of the problem that I can solve
with the smallest program. Then I start adding things.

ORN: Looking at Neal Stephenson's (admittedly fictional) Cryptonomicon and George Dyson's
(serious history) research on early hackers, it seems that computing started
out as hacking, as you describe it: bursts of creative energy solving
interesting problems.

If it's fair to talk about hacking as "creating something beautiful", how
do you recognize beauty in code? What makes software beautiful?

PG: A good but very hard question. One way to answer it
would be to think about what makes code ugly. E.g. gratuitous cruft,
lack of generality. Beautiful code would thus be succinct and general.

ORN: Where might a student find beautiful code to read?

PG: Fortunately, that is a lot easier thanks to the open
source movement. I suspect if you start reading the source of programs whose
design you admire, you'll find the code is good too.

ORN: Is there room in the business world for beautiful
code? Does beauty come into conflict with more pragmatic concerns, or does
beautiful code have a fitness advantage?

PG: Yes, there certainly is room in the business world for
beautiful code. For example, one of the criteria of beauty is brevity. Bugs
increase more than linearly with the size of a program. So a shorter program
will also have the very practical benefit of having fewer bugs.

ORN: How could programmers under deadline pressures explain
this to non-programmers? I've had the experience of being told to skip
disciplined programming in order to finish the code. Is this because people
don't appreciate that code can and should be elegant?

PG: Sometimes it does take longer to write beautiful code.
If you really don't have time to write good code, then I suppose you have to
write bad code. But watch out for using this as an excuse. And remember that
for anything beyond a small hack, it may mean a net savings in time to do
things properly, just as it is often a net win to clean your kitchen before
cooking something.

ORN: Combining ideas from Why Nerds Are Unpopular--that schools exist in part to keep children out of the way of adults; smart kids
ignore the social games that arise in place of things that actually matter--with ideas from "Hackers and Painters"--becoming a master
craftsman takes years of work and practice and refinement; "software engineering" education misses the point. Are you envisioning a system of mentoring and apprenticeship?

PG: I think you learn to program mostly by programming.
The most a teacher can do is suggest interesting paths -- e.g. by setting
interesting problems to solve, or by describing mistakes other people have
made. And in good CS departments, at least, this is how things work now.

ORN: In my experience, there seems to be two different
philosophies guiding language design these days. One philosophy comes from
mathematicians. You'll find these languages exploring static-type systems,
formal provability, and the like (though not all at once). Another philosophy
approaches programming from the human level, saying that things can be messy,
compilers can be complex, and even, in some cases, that it's better to emulate
natural languages than to pursue purity and extreme simplicity.

"I have a hunch that the main branches of the evolutionary tree
pass through the languages that have the smallest, cleanest cores. The more of
a language you can write in itself, the better."

Would you classify yourself as sympathizing with the mathemeticians or the
humanists?

PG: I think there is a middle path. It's good for
languages to be inspired by math, but they have to be inspired in the right way
by it. What language designers don't seem to realize is that mathematicians do
two things. They prove theorems, but they also invent notation. It's the
second aspect of math that language designers ought to be looking at. They should be asking not "How can I prove theorems about programs?" but "How can I invent something as fertile as place-value (i.e. "arabic") notation?"

ORN: That sounds a lot like layers of abstraction. Is that
how McCarthy approached Lisp (or how you're approaching Arc)?

PG: Yes, McCarthy (as far as I know) invented the
axiomatic approach to programming language design, and this is the way I've
been approaching Arc too.

ORN: Have you had the experience of inventing a notation
that makes a seemingly unrelated and intractable problem suddenly much easier?

PG: Some of the nondeterministic choice operators I built
on top of continuations in On
Lisp had that flavor. I doubt I invented them, though.

You get two operators, choose and fail. Choose takes a
list of alternatives, and returns one that doesn't later
result in a call to fail. Behind the scenes, it is all
done by backtracking search, but this is hidden in your
program. As long as your code is purely functional (i.e.
has no side effects) you can act as if choose had foresight.
The effect in source code is rather magical.