Growing with Gentoo

When Daniel Robbins recently spoke to 30 or so students and professors at
Stanford's Computer Systems Lab, he didn't have to waste
much time explaining what his baby, Gentoo Linux, was. Most of the audience
seemed to have at least one Gentoo playbox each. And those who gathered around the
project's founding father after the talk seemed pretty happy with how Daniel's
distribution has been shaping up.

Instead of going into the details of Gentoo that have made it so
popular -- the distribution's preference for source packages compiled by the user over
binary packages, its ultra-configurability, the heavy automation of Gentoo's
emerge build system -- Daniel sat down and spoke on the lesser known aspects of
Gentoo: where it came from, what problems and opportunities Daniel was seeing
these days, and his opinions on where the future might take it.

Gentoo's genesis was, like so much open source software, as academic as
the talks' setting. Robbins was working as a system administrator for the School of
Medicine at the University of New Mexico when he first started playing around with
Linux.

He began with Debian, later moving on to work with others on developing
an optimized distribution for the Stampede project. Stampede was an open
source endeavor that suffered from personality clashes and, says Robbins, a
poor management approach that did nothing to allay those problems. Robbins
decided to move on (Stampede is no longer maintained), and started working on rolling
out his own distribution.

A lot of Gentoo's philosophy and appeal comes from this time, Robbins
says. Portage, Gentoo's famously automated build system, "was necessary,
because I was the only person working on Gentoo, and I had time only to do
everything once."

Linux/Unix System Administration Certification-- Would you like to polish your system
administration skills online and receive credit from the University of Illinois? Learn how to administer Linux/Unix systems and gain real experience with a root access account. The four-course series covers the Unix file system, networking, Unix services, and scripting.
It's all at the O'Reilly Learning Lab.

Robbins also set out to make the distribution's development as open as
possible, to prevent the short-circuiting of command that beleaguered
Stampede.

These days, Gentoo is a little bigger than just Robbins. The ebuild
tree holds 7,000 packages. Development and support is spread across
dozens of mailing lists and IRC channels (with the main list, gentoo-users,
receiving in excess of 100 messages per day). Robbins guesstimates that over
50,000 people have a Gentoo installation. ("We have no figures.
It's over a thousand, I know that.")

With such an explosion in the size of the project, Robbins admits that
the distribution is struggling to adjust. "I delegate. But the biggest
problem is finding enough time to even delegate."

Much of his design is coping well with the shift in scale, and those
elements that do not are, he says, receiving attention; USE flags, for
instance. USE flags determine high-level, global compilation settings in Gentoo, such as
whether all packages should be compiled with GNOME support, or with SSL
turned on. With so many packages contributed, the list of USE flags has
mushroomed in size. Robbins is looking at ways to turn this into a hierarchical
list.

The biggest problem, though, is deciding in an open manner which
direction Gentoo should go.

Gentoo has always worked on what Robbins describes as a minimally
bureaucratic system. Gentoo developers, for instance, have CVS access to the whole
Gentoo tree, allowing them to tweak packages that are not their own primary
concern.

Fitting users into this model has been difficult, because until
recently the stereotypical Gentoo user was a developer. And coordinating the
future development of key utilities (like emerge, Gentoo's build utility, and
catalyst, its distribution creator) with so many offering suggestions
has grown impossible.

But Robbins has made a conscious decision to avoid the political
governance model of Debian, which revolves around elected officials and fixed,
voted-on policy. Instead, he's been looking at informal frameworks like Python's
PEP system, which has an open period of solicitation for new feature
suggestions from users, and then a closed, fixed time period -- this is when the
main Gentoo developers work on implementing what has been decided.

Perhaps as a nod to his audience, Robbins also admits that Gentoo could
do with some traditional project management. "There are a lot of us [in
open source] who don't have formal software development skills. There's an
arrogance that comes from getting something as big as this to work,
and it can be bruising to one's pride [to admit that we need to know more] ...
that we should have paid more attention to that class in college."

But Robbins is most concerned that Gentoo preserves its reputation as
the most open and flexible of distributions. When asked whether he thinks
Gentoo's size will lead him into making it as inflexible as other distributions,
Robbins answer is simple: Gentoo's developers won't let him.

Gentoo has other reputations, as well. Some are old and outdated -- you no
longer have to spend a week compiling glibc before you can get anything
to work. New customized installs (called "Stage 2" and "Stage 3") can let
you skip the most time-consuming builds, or even skip to a complete
pre-compiled suite of the most common packages.

Others have a germ of truth to them. Robbins acknowledged twice in his
talk that Gentoo users have a reputation for pestering upstream open source
developers with bug reports. Some have been legitimate -- the idiosyncratic
configurations permitted by Gentoo often shook out obscure problems in
the most stable packages. There's a general feeling among some developers
that Gentoo users are identifying problems caused not by upstream bugs, but
by aggressive optimization or other poor configuration choices that the
users themselves have made.

Robbins wants to stop this, and emphasizes that if Gentoo users have a
problem, reporting bugs upstream to the software's original author is
the wrong place to start, and that they should let the Gentoo development
team know first.

You sense, listening to Robbins (and hearing him listen to others after
the talk), that the aspect of Gentoo that he is striving hardest to
preserve is the sense that there is always somebody you can reach to help you solve
your problem. This is what Robbins believes lies at the heart of Gentoo's
appeal.

"We started out as an IRC project ... I'd release it, and people would
give me instant feedback on bugs. No formal release cycle, just instant
development. I really enjoyed getting feedback, and then fixing it and saying 'here you go.'"

"This is the heart of Gentoo: a technology where if someone doesn't
like how it works, we can change it, or teach them how they can change it
themselves and make their tool work for them. The heart of Gentoo is that concern
for others, and the joy we had helping others."

And what of the future? Daniel keeps diplomatically quiet on most of the
fine points, but nods at the barrage of suggestions from the Stanford
audience. One wants cross-compilation so he can build Gentoo for his older Sparc
systems without it taking weeks to create. Daniel says that cross-compilation
of a sort will be in 2004.1, the version due to be released next month.
"It's aimed primarily at embedded systems developers so they can create a
distribution on the host machine, then transfer it over."

GPG-signing is another request. Gentoo etree releases are sometimes MD5-signed, but the system as a whole isn't necessarily secure (a hostile
attacker could fake the MD5s as easily as insert new code). With an eye to
Gentoo's increasing use in the enterprise, Daniel says this will be another
incoming feature.

Whatever happens next to Gentoo, Robbins' place at the center of it
seems assured, at least. He revealed that he has a Gentoo-inspired
salary these days, thanks to an anonymous German benefactor who called him late one
night and offered him money to continue hacking Gentoo.

Even without the financial help, you get the feeling he'd carry on
regardless. And when one fan tells him that Gentoo has "been the most fun I've had
with Linux since 1996," Robbins looks like he can only agree.