Free Clues from Eric

In mid-October, Eric S. Raymond's book
The Cathedral and the Bazaar came out from
O'Reilly & Associates. It contains updated versions not only of
Eric's landmark title essay, but also both companion pieces,
“Homesteading the Noösphere” and “The Magic
Cauldron”.

We thought it would be a good idea to interview Eric about
the new material in his book—and, since this historian of the
hacker tribe never stops thinking, to glean a few of the ideas
which have evolved out of his brain since finishing it. This
conversation took place on September 22, 1999.

Doc: So now that we have
“The Magic Cauldron” in final form, what new ideas are you
working on?

Eric: A couple. I'm further
developing the critique of conventional software-development
management that I outline in the new last section of “The
Cathedral and the Bazaar”. Also, I'm noticing some interesting
developments in the business ecology of Linux.

Doc: Let's take the
development management idea first.

Eric: I've been developing
my thoughts about play being the most efficient form of work. Let's
start by asking a basic question: What are the circumstances that
make a programmer productive?

A programmer is productive when he is neither
under-challenged nor over-challenged—neither bored and
underutilized at one end of the scale, nor burdened with artificial
deadlines and technical and procedural hassles at the other end.
When the programmer's skill and energy are well-matched to the
task, that's when you'll see the maximum productivity. The
interesting insight is that these are also exactly the
circumstances under which the programmer will be happiest, when his
enjoyment in writing good code is maximized.

So in order to get the maximum economic leverage out of the
guys you hire, you need to reduce friction costs at the top end,
because you'd rather pay for their code than their grief. On the
other hand, you don't want them to be bored—because when they're
bored, you're not extracting maximum value from their
ability.

The general prediction is this: enjoyment predicts
efficiency—and not just in some fuzzy New-Agey way, but in a
hard-nosed economic sense.

If you use your developers the most efficient way you can in
creative work, they will be happy. On the other hand, if you have a
development management style or an institutional tradition under
which your developers are bored, frustrated and angry all the time,
you are failing. And not just humanistically, but economically. You
are failing to extract maximum productive value out of your
people.

Doc: You have Dilbert's work
environment.

Eric: Right. If you look at
traditional software development management, you'll find that of
the five classic tasks of the manager, maybe four just don't apply
at all in an Internet environment. It's not so much that the
traditional assumptions are wrong in some deep sense, but they're
irrelevant: they proceed from assumptions that no longer match the
problem. Because the assumptions don't match the problem, we've got
an impedance mismatch: a lot of bored, frustrated, angry and
unproductive programmers out there.

Now there is a popular “burger-flipper” school of
management that says this kind of unhappiness in production workers
doesn't matter as long as they're still cranking out a minimally
acceptable product on the right deadlines. In fact, some managers
just sort of gut-level assume that people aren't working hard
enough unless they're stressed out, so they figure the right thing
to do is apply the whip every once in a while just on general
principles. This is the thinking behind artificial deadlines—it's
the Simon Legree theory of motivation.

But a funny thing happened on the way to the plantation—the
overseers themselves are revolting. Managers are increasingly
frustrated, because they're doing all the supposedly “right”
things, and their jobs and their results still suck. The days that
only engineers griped about this stuff are long past. Today,
Dilbert cartoons hang in executives' cubicles.

Think about that. If the system was working, would the
managers display contempt for it?

Doc: Tom Peters has observed
that the best-selling business books are his and Scott Adams'. And
he laments that all those Dilbert books are relentlessly
cynical.

Eric: I'm totally with
Peters on that one. When I speak to CEOs and investment bankers, I
try to wake them up to the fact that the popularity of the Adams
books all the way up the management chain is actually not funny. It
means the system is broken. If we're going to fix it, if we're
going to get our productivity to where it ought to be, we've got to
start thinking humanistically about what makes people happy at
work. We've got to banish the Dilbert syndrome.

Doc: What about this ecology
thing?

Eric: What I'm noticing
lately is that Red Hat has developed a couple of satellite
distributions that are significant in their own right. One of these
is Kevin Fenzi's Red Hat Über Distribution (KRUD)—which I am
now running on my own machine, by the way. The other is
Mandrake.

Doc: These are cases where
others are leveraging Red Hat.

Eric: Right. Both looked at
the Red Hat distribution and said, “I see a significant value-add
here that Red Hat is either not interested in or not in a position
to exploit.”

In the case of KRUD, Red Hat doesn't put out updates often
enough. So what Kevin does is press CD-ROMs every month on a
subscription basis. A new one shows up in the mail, you stick it in
your drive, go through the update procedure that pulls all the new
RPMs off the CD-ROM and installs them, and you are completely up to
date.

I think of this as the magazine model of operating system
publishing. Want all the latest updates? Subscribe—it's just $36 a
year. In return, you don't have to spend a lot of time downloading
stuff and worrying if you're current.

Doc: Who's the
customer?

Eric: KRUD is particularly
valuable if you are a site with serious security requirements. Want
to be sure you have all the latest security patches? That's what
Kevin does for you. He's the maintainer of the Linux Security
HOWTO. He's at
http://www.tummy.com/, by
the way.

Doc: Why can't Red Hat do
this?

Eric: Scale, I think. They
have to ship CD-ROMs in sufficiently large runs to carry their
other expenses. They would be bleeding all over the place in
fulfillment and marketing expenses if they tried to compete in this
new space by going to a one-month rather than a six-month release
cycle.

Doc: Red Hat only scales
higher up the value chain.

Eric: Right. Kevin can do
this because it's a relatively small effort for him, and he doesn't
carry all of Red Hat's development and marketing overhead.

Doc: What about
Mandrake?

Eric: Their value-add is
that most people have Pentiums now, and Mandrake can take all the
source RPMs and recompile them with Pentium optimization so your
machine runs 30% faster. Again, it would be too expensive for Red
Hat to have two distributions, one optimized and one not.

Doc: Does Red Hat benefit
here?

Eric: Sure. Red Hat is
certainly growing their support market this way. If I'm a Fortune
500 site and I have a subscription to KRUD and have a support
requirement, do I go back to Kevin? No, because I know he's just
one guy sitting in a room in Colorado somewhere. So I go back and
buy a support contract from Red Hat.

Doc: And if you're in the
same position with Mandrake?

Eric: You may go to
LinuxCare. I believe they're Mandrake's partner for service. So the
Mandrake/LinuxCare combination is competing with Red Hat a little
more directly for service revenue. Still, the Mandrake guys have a
good working relationship with the Red Hat guys. In effect,
Mandrake is acting as an R&D arm for Red Hat, because the
changes they make, to fix and clean up things, can get folded back
into future Red Hat distributions.

Doc: And the ecology
point?

Eric: Because everybody is
playing by the open-source rules rather than competition, there's a
potential for cooperation to make more money. There is a potential
for single distributions to speciate into mini-ecologies of
mutually supporting distribution variants. These guys all share
core DNA, and that's what makes the ecology work.

Doc: I imagine there is some
analog in original markets.

Eric: Oh, sure, it's a value
network in Christensen's sense. [Clayton Christensen is the author
of The Innovator's Dilemma, an award-winning
book on disruptive change in technology markets—Editor]

Doc: Have you got any more
books in the pipeline?

Eric: Yes. I'm about halfway
through writing the first draft of The Art of UNIX
Programming, a book on how to think like a UNIX guru.
It's aimed at a lot of the younger Linux programmers who have
picked up bits and pieces of the UNIX design tradition, but don't
really have the whole Zen. This is not their fault—because like
Zen, it's only been passed along from master to student up to
now—“a special transmission, outside the scriptures”. It's time
to write it down.

Doc: Any other significant
developments for you?

Eric: I'm doing most of my
programming in Python these days. I've moved away from C.

Doc: Python is hot. It's
taking our own people by storm.

Eric: Under present economic
conditions, using a language in which manual memory management is
required almost never makes sense. The thing about C is that you
have to manage your buffers and dynamic storage by yourself. You
don't have a garbage-collecting interpreter doing it for you. It's
a huge source of problems—classically, something like nine out of
ten of the runtime errors in C programs are memory-management
screwups.

It used to be, back when machines were more expensive and
programmer time less so, that it was worth hand-tuning resource
utilization and putting in all the debugging time necessary to
compensate for the high error rate. But unless you're doing systems
programming or really heavy-duty number crunching, that tradeoff
just doesn't make sense any more—not with cycles and memory as
cheap as they are now.

Doc: Is that Python's main
virtue for you?

Eric: One of several, but in
my opinion it is the single most important one.

Doc: But other languages do
the same thing.

Eric: Yes, there are lots of
interpretive languages that do garbage collection, but very few are
as well-designed as Python. The alternatives don't scale well; they
make it harder to read and modify large volumes of code. I used to
write a lot of Perl, until it got too painful. And the less said
about Tcl, the better.

Doc: Closing
thoughts?

Eric: Just a sound-bite
version of what's beneath most of the open-source business models
we're seeing out there. “Linux is free. Clues are not.”