Interview with Gentoo’s Daniel Robbins

Gentoo is so far, the big Linux surprise this year. With its 1.0 release took the Linux world by storm and converted a huge number of power users and developers from the well-known Linux distros they were using, to the lightweight Gentoo Linux. While its installation process is not for the faint of heart, it pays back the user with a highly optimized system. As a result, Gentoo is dubbed the “fastest Linux distro” to date. Read on for an exclusive interview with Gentoo’s project leader, Daniel Robbins where he reveals that Gentoo will be further optimized with the fastest x86 C/C++ compiler (Intel’s ICC) in addition to GCC 3.1. Daniel also speaks about the future plans for Portage and the overall system in general.1. Gentoo 1.0 has become a huge success! How do you feel by seeing all this hard work coming to a climax with the thousands of downloads and all the talk around it?

Daniel Robbins: As project lead, it has been very satisfying. People are understanding and embracing our vision of how Linux distributions should be designed, which is very satisfying. It’s also made my life quite hectic, but hopefully I’ll figure out how to adjust to the new situation that I find myself in 🙂

2. Without doubt, the two reasons users like Gentoo is its speed and Portage. Are there plans and work being done for a full move to GCC 3.1 or even ICC? (both support Pentium4 optimizations among other goodies)

Daniel Robbins: Yes, we are planning to move to gcc 3.1 as we’re able. Many people are already using gcc 3.0-based Gentoo Linux systems. We also have one developer (William McArthur) working almost exclusively on icc support and making great progress. He has been actively reporting bugs to the Intel compiler team and getting ready for icc6. Sometimes, it seems that the Intel compiler developers aren’t as interested in fixing reported bugs as they should be, and this tends to slow William down a bit. But all in all, we are making very good progress with icc and gradually getting ready for gcc 3.1 too.

3. Portage has come a long way, but it seems to be in a state of flux. What are the plans for Portage’s future? Which problems need to be addressed? Is Portage 2.x going to be written in Python as well?

Daniel Robbins: Yes, we need to continue to use Python so that we can leverage our relatively miniscule coding resources which are currently me, Jon Nelson (< plug > maintainer of Boa, the fastest userspace Web server < /plug > ) and occasional patches from other members of our dev team. The RAD aspects of Python are essential for what we are trying to do and its “openness” (the fact that it is a scripting language and has a clean syntax) makes Portage development accessible for others who want to contribute.

As far as problems that need to be addressed, we need to improve our error handling, move to a unified high-performance (yet lightweight) database backend, improve the performance of our current text-based database code, and add bunches of miscellaneous features all over the place in order to make our feature set more refined. “Refinement” is the key word here. We have always been extremely responsive to user requests for new features, and I plan to continue our user-centric development style. Portage is designed to make peoples’ lives easier, allowing them to be in full control of Linux technology rather than being overwhelmed by it or forcing them to turn to a third party (the Linux distribution creator) to make decisions for them. Our users let us know what they would like to see, and I do my best to keep them happy.

4. How is Gentoo, the company, organized? What is its member structure? Also, how one can get CVS commit access?

Daniel Robbins: At its heart, Gentoo Linux really isn’t a company but a development team and user community. I do have a corporation called Gentoo Technologies, Inc. that holds the copyrights for the vast majority of our GPL code, but that’s it. We aren’t generating any income from Gentoo Linux (besides donations), and our development team is 100% volunteer. Generally, we have been completely supported by donations, particularly from a few of our developers. For example, our server is in a great datacenter thanks to a generous developer. As we grow, we plan to gradually wean ourselves from our dependence on donations by developing creative and “free software-friendly” ways of generating income.

5. Gentoo’s bleeding edge features, sometimes make users… bleed over bugs of app/library versions that are not yet truly ready for larger public consumption. How is this issue going to be addressed?

Daniel Robbins: You’re absolutely right, Gentoo Linux is currently a “bleeding-edge” type distro. It makes Gentoo Linux the ideal distribution for hobbyists who get lots of cool toys before everyone else, but makes Gentoo Linux a questionable choice for production server environments, especiallly if you’re a newbie who has the insatiable desire to keep your server continually up-to-date. Of course, Gentoo Linux is my OS of choice for everything, but I also know what I’m doing. Newbies should know that sooner or later, “emerge –update world” will cause something to break due to a developer error or unforseen incompatibility, and it will require a little manual attention in order to get working again. For this reason, we recommend that those using Gentoo Linux for servers avoid updating packages unless absolutely needed, and that newbies use something like FreeBSD, Debian or Slackware for their server needs, at least for the next six months. They’re much more boring, which can be a good thing 😉

Obviously, the fact that we aren’t an ideal choice as a server is a *huge* concern for me and for the vast majority of our development team. Some people think that because we are popular, we must be hapy with where Gentoo Linux is now. The fact is that while we appreciate our current popularity, most of our dev team is very perfectionistic in nature and we really feel that we have a ways to go before we have realized the vision of what Gentoo Linux is all about. Gentoo Linux is really only in its formative stages — what you have seen so far is only a taste of our vision. There’s a reason why the current version of Gentoo Linux is “1.1a” 🙂

OK, that being said — are we going to address this issue soon? Yes. How? We’re going to take pieces of the current “bleeding-edge” Gentoo Linux meta-distribution, refined them and use them as the basis for a robust, well-maintained version of Gentoo Linux — geared exclusively for servers. For this project, we will reduce the number of ebuilds in our server branch from 1800 to around 400, at least initially. Our stable CVS tree will be completely separate from our current bleeding-edge version — a “code firewall”, if you will. Commit access will be limited to an elite team of Gentoo Linux developers. We will lock down upgrades so that “emerge –update world” will only fix known bugs and security fixes. Each release of this new server meta-distribution will have an official one-year lifespan, during which it will be painstakingly maintained by us. In-place upgrades to new releases will be fully-tested and very smooth. We will have some cross-pollination with our current tree, but anything that goes into the server distro will be carefully audited before being added. We are still developing the goals for our new server project, but based on feedback from the rest of our development team (who seem to be in near unanimous agreement) it looks like the project will progress very closely if not identically to how it is described above.

6. Is Gentoo going to sell its own ISO CDs, and become a fully commercial company, or are you going to stay largely community-based?

Daniel Robbins: Hopefully something in-between. I never want to compromise our free software development principles in order to make money. Nor do I want to put our project in a position where we are *tempted* to compromise our principles in order to make money, as many companies are. We certainly don’t want to try to fit into the mold of companies like Red Hat and Mandrake just because that’s the “distro way”. We want to choose our own path and find a healthy synergy between developing free software and growing a business based around Gentoo Linux.

I very much want to find a way to turn the Gentoo Linux project into a profitable enterprise. My main motivation in wanting to do this is so I can stop living from paycheck to paycheck and focus my professional efforts exclusively on Gentoo Linux development. Many of our developers would like to do the same thing.

Right now, we’re in a catch-22 where our sudden popularity has put us in a position where we *could* probably start generating some income and I and possibly other developers could potentially start working on Gentoo Linux full-time. Yet our sudden popularity has also overwhelmed us and eaten up all our time — time that we would otherwise be using to build a business around Gentoo Linux. So we’re basically holding tight until we get the time and resources to look into the whole business thing.

7. Are there plans to make the installation more comprehensive for newbies and Linux agnostics, or your target will still remain power users and developers?

Daniel Robbins: Yes; there is a Linux distribution using Gentoo Linux as a base for its efforts and they have developed an anaconda-based installer for Gentoo Linux. There will undoubtedly be further development in this area; while it’s not a primary focus of our development efforts, it will certainly happen as we proceed towards Gentoo Linux 2.0. But we will always have a clean, “bare-bones” shell-based install method so that the user has complete control over how Gentoo Linux is set up if he/she so chooses. We’ll *always* have that.

8. Gentoo comes by default with a specially patched kernel. Please tell us about the speed increase users gain with Gentoo when compared to stock kernels that other distros include.

Daniel Robbins: People are finding our kernel to be quite zippy, particularly when it comes to interactive responsiveness — something I care very much about. It’s based on -ac so it has rmap and O(1) as well as some new IDE stuff. We’ve also added Robert Love’s preempt patch, Andrew Morton’s low latency code (both can be turned on/off in “make menuconfig” as desired) and bumped the HZ on i386 systems to 1000. Thanks to the efficiency of O(1) and rmap, HZ=1000 works nicely to increase responsiveness without introducing any significant overhead. And of course preempt does amazing things for interactive use — much more so than low-latency. It’s clear that good interactive response isn’t *purely* about low-latency — otherwise Andrew Morton’s low latency patches would give much better and crisper interactive response, which isn’t the case. Preempt seems to make the timeslicing of the CPU much more uniform (this is an educated guess, not a fact), which in turn appears to make a huge improvement in the “feel” of everything. It makes everything feel much more Amiga-like and presumably more BeOS-like.

Besides our performance-related improvements, we also now include XFS 1.1, htb2 (a very good QoS patch), grsecurity and a handful of patches from Jörg Prante’s bag of goodies, including freeswan (ipsec) and the international crypto patch (only available in our “crypto-sources” kernel which is an install option; our standard “gentoo-sources” kernel doesn’t have any strong crypto so that people in countries where strong crypto isn’t illegal won’t break their country’s laws unintentionally.) We also include Andrea Archangeli’s 3.5Gb address space patch and a recent snapshot of the ACPI code being developed by Intel.

9. Is Gentoo… bleeding edge enough to make the big step and include a 2.5.x kernel as an additional option to the stable 2.4.x one?

Daniel Robbins: Sure. We don’t have that as an option now because I don’t use 2.5 and I’m the guy who rolls up the kernels. But adding it wouldn’t be a problem. Is that a personal request? 🙂

10. What are the future plans for Gentoo? What architectural and other changes do you have in store for the new version and when this new version is expected to be released?

Daniel Robbins: Well, Gentoo Linux/PPC is taking off and despite its relative infancy is apparently the best thing out there in the Linux/PPC world, from what I hear. I haven’t been able to run it myself since I don’t have a PowerPC system, but I may be able to afford to get an iBook in the near future and start playing. We have a very enthusiastic and dynamic PPC developer and user community, which is exciting to see 🙂

Our Sparc and Sparc64 ports are also progressing very nicely. Our Sparc user community is still very small but should be growing significantly over the next year, especially as we start focusing more on building a rugged server meta-distribution.

Beyond that, I would really like to port Gentoo Linux to AMD’s new 64-bit platform. After that, probably Alpha and then Itanium.

In the next year, you will start to see Gentoo Linux development become more server-focused with the birth of a new version of Gentoo Linux designed the ultimate ports-based Linux server toolkit.

Our selection of kernels will grow — we currently offer Red Hat kernels and will also start offering Mandrake (stable and cooker) kernels as well. You’ll see gcc 3.1 support and possibly a high-performance icc-based version of Gentoo Linux. Portage will improve dramatically in performance and functionality. The number and quality of ports will improve. We will begin offering products and services that complement but do not compete with our free software development efforts.

Right now, many people consider Gentoo Linux to be the most exciting and desireable “bleeding edge” distro out there. In a year, we’ll be well on our way in establishing Gentoo Linux/server as the ideal server platform. It should be a very exciting time 🙂

About The Author

32 Comments

Yes, I know it is YAFLD (Yet Another Fantastic Linux Distribution – at least I think the F is “Fantastic…”), however it does break the mould.

I’ve always though it good to compile from source using either a Slackware or Debian setup, however Gentoo (and its port system) is a very classy setup & well polished.

2002-05-14 8:22 am

Too bad it doesn’t recognize my NIC and I don’t know how to add the driver..

2002-05-14 9:45 am

There are other sourcebased Distros like SourceMage,Sorcerer and others that are as fast with gcc optimisations. And their package system is much simpler ( just scripts in bash ). Writing your own ebuilds is much more work. They also have easy installation and everything you need. SourceMage even has a stable, test and a developers branch!

2002-05-14 10:19 am

… give me spare time, bandwidth and disk space to give it a try on my iBook

2002-05-14 12:11 pm

This kernel sounds like it would be a welcome addition to any existing Linux installation, not just Gentoo. Where would one go about getting their patched kernel?

I have been watching gentoo with interest, but installing Debian onto my laptop was enough work for now. However, I’d love to use some of the work that Gentoo has done… but there’s no real documentation or instructions on how to patch a standard kernel to bring it to the kernel used by Gentoo. Can any current Gentoo users explain how I would go about doing this?

2002-05-14 1:20 pm

As a follow-up to my own posting, download the linux-2.4.18.tar.bz2 package:

This will bring the 2.4.18 up to Gentoo’s newest kernel. Which, FWIW, has the needed ac97 patch for my Presario 700 already included. I’m recompiling right now, and look forward to O(1), preemptive, as well as a kernel with the needed ACPI and ac97 patches already included. Sweet.

2002-05-14 1:37 pm

I’m a relative newbie to the Linux world. I installed Gentoo after about three attempts (learning something with each atempt). It works beautifully. Their instruction is well written and comprehensive. I took it step by step and I have a working Gentoo system that I’m very happy with.

I attempted Sorceror and a couple other streamlined distros and got nowhere. I thought Gentooo relatively easy and a great learning process.

2002-05-14 1:39 pm

I put Woody on my Pavillion zt 1130, but the sucker ran so damn hot becasue the 2.4 kernel does not have support for the Advanced power controls. 2.5.x does not either…..I guess I will just have to wait.

my next Laptop will be a Powerbook though….1 yr. 9 months……..can’t wait.

2002-05-14 2:45 pm

Robert: Gentoo wont be working on your ibook, not any time soon anyways. Gentoo is x86 specific, which i find to be a dissapointment. I really dont understand why such a distro is x86 only, its not like GCC doesnt exist for the other architectures.

2002-05-14 3:16 pm

If you read the interview as well as check out their webpage you would see that they are currently working on a PowerPC port and a Sparc/Sparc64 port, when these are finished there is talk of porting it to the Alpha and probaly to the Itanic (IA64)

Richard, you’re wrong – if you had read the whole article you would have found an answer to PPC support in question 10.

Btw, what do you expect – that a x86 user buys a PPC just to make his software available in binary form to PPC users?

2002-05-14 3:49 pm

I tried to install it but couldn’t find module for network card Marconix (is it tulip?) well I was quite sad. I started to think about sorcerer or something like source mage… But since I noticed there is distro based on Gentoo with anaconda-based installer I want to know where to get it. So if someone knows please post reply. It’s a bit hard for newbie. And I saw I should learn Fdisk… a got a bit unexcited, I really don’t have time to go so deep, but I like my system being fast so if you have any suggestion please reply.

2002-05-14 4:18 pm

Ummmm…

ebuilds (portage’s package definitions) are overglorified shell scripts using bash functions and an object oriented like approach called eclasses to share code between ebuilds.

ebuilds will be a tad harder to develop then something as simple as sourcer because of the USE settings system and the fact the installation process is sandboxed to prevent unauthorized writes to your real filesystem.

2002-05-14 5:40 pm

Be warned though: I’ve run into some trouble with updating and things not compling out of portage, although I fully admit this may be because I’ve been updating sproadically since 1.0-rc6-r15. Things I’ve had problems with since last night when I rsync’d:

– sudo (although that’s been fixed)

– you have to edit the samba ebuild to remove quotas (this is more a linux problem, even the samba .h file says it)

– something else in libtif broke last night

– you have to hand merge your /etc files after an update.

– where my nvidia glx drivers once work, the glx part broke on a previous upgrade.

I like gentoo (or else why would I be running it? ), but it needs to polish some edges before I’d run it on a machine that I couldn’t wipe out on a moments notice. Of course I feel that way about Linux in general.

support for cpu clock/voltage scaling will be in 2.5 (if it isn’t already).

Scroll down to the start of the list of beta additions. Sadly the link doesn’t go anywhere useful. But at least it’s coming.

2002-05-14 7:15 pm

I am using gentoo, not very long, since the release of 1.1a.

I think gentoo is the best distribution i have ever seen. I am developing myself (c++, most of the time console), and i only can say gentoo is THE distribution! I simply love the ease to use portage system. I always loved the BSD-Portage system.

However, i am happy with gentoo, and i don’t think on changing to an other distribution in my dreams!

I havn’t had really problems while installation, only with libpng, because the wrong libpng was in portage, they fixed that now…

Some problems with my printer and like that. But now everything GREAT!

I only can thank the gentoo-developers for bringing up a revolutionary system like that.

2002-05-14 11:14 pm

I’ve only been using Gentoo for about a month now (embarrasingly enough since I read about it in Leo Laporte’s blogg) and I thought that I would chime in with a perspective that is biased towards the Debian users (I have been a big Debian user since 2.0 back in 1998).

Putting aside all of the bleeding edge compiler optimization crap — which is mostly BS, so dont be dissapointed when your old PII 400Mhz doesnt run 10x faster– Gentoo is great for two reasons:

1. most of the stuff is much more out in the open than Debian. Portage for all of the hype is simply a well organized mix of python-rsync-bash, a mix which in my opinion took a touch of art a touch of experience and a touch of genius. It is far far easier to get under the hood of than is apt, dselect or dpkg. How many times have you done something in dselect only to have some dependency conflict cause the system to suggest that you uninstall base-perl or something stupid like that? With Debian you can always back out, but I found that I would often not end up understanding exactly what had caused the problem in the first place. With Gentoo, all of this is much easier to diagnose. You end up learning more about the system, the packages and just how powerful a few basic UNIX tools can be when used together in the right way.

2. Gentoo is younger and therefore more approachable. Debian was probably like this before I joined the community in 98, but by the time VA linux was in full steam (99) approachinf Debian developers at conferences was like approaching Apple employees at MacWorld– an exercise in restraint to say the least. All projects with success go this route of arrogance in the long term, but gentoo is fresh for the moment and that makes it a fun place to hang out.

That said, Gentoo is not without its problems. A couple of weeks ago, there was a decision to stop supporting XFS in the main bleeding-features kernel that has left me with a laptop install that is going to need some serious fixing. My attempts to install the most basic of services on the first server I built, mail (qmail), bombs during the installation and has not been fixed. Their rsync mirror system seems to be slow to propagate changes.

Still, if you have more than a passing interest in linux distros and want to get in on the ground floor (more or less), Gentoo can’t be beat. Its the most fun I’ve personally had the halcyon days of lattes and changing the world that the web brought us.

2002-05-14 11:21 pm

I’ll stick my hand up as someone else who’d like to try gentoo, but is prevented by my network card (netgear FA311) not being supported by the install

2002-05-15 3:34 am

Hi,

I am using gentoo since 1.0

A migrated from slackware and I am loving gentoo!

I wrote down the installation-steps from the manual and followed them. I had no problem and some hours later I ha a complete system with kde3.

I had some problems with kde3 because of a typo (a jed did not shown an ascii-char…).

I am updating gentoo on a daily basis and didn’t run in any problems, despite of a (solved) libpng version conflict.

The mailing lists a cool, a lot of high quality traffic!

Glück Auf,

Volker

2002-05-15 5:27 am

Just a quickie, for all you guys who expressed their interest but complain about the nic that is not supported by install CD. Since this issue was mentioned something like 3 times out of 20 posts I decided to throw in a short comment.

First, you do not have to use install CD. Gentoo does not use any fancy installer, you get directly to commmand prompt right after the CD is booted. This was done on purpose: this allows maximum flexibility and makes it possible to use almost any method to kick-start your installation. Here are just a few:

1. The best one. If you have any kind of linux installed on the system (that will probably support your hardware already) you can just skip to create/mount partitions and go on from that point. Please take close look at the FAQ and various install instructions awailable from

This is the best since you will be able to use your old distribution while you are building your fresh gentoo system. All the way through! That is you just reboot in the end and there you are!

2. You can get any boot disk, boot off that and go on as in previous case. Just make sure this boot disk contains whatever drivers are necessary to start you up.

3. You cannot find such disk? (Don’t you have a resque floppy? Don’t tell me you did not create one..) Still all is not lost. You can take probably more troublesome but most educating route . Download install CD and disassemble it. Replace provided kernel with the one you compile yourself, just make sure to compile in (or as a module) all necessary drivers. Assemble the CD and use it! Gentoo uses isolinux to create bootable CD. Try searching mailing lists. Also you will find help on forums and irc (#gentoo at irc.openprojects.net).

George

2002-05-15 9:22 am

Sorry but I got my mother to install gentoo =), saying that this is not a newbie’s distro is not well put. This is one of the better ways to learn, sure it takes time and alot of coffee. But you didn’t learn to tie your shoes in 1 day did you?

2002-05-15 3:54 pm

Netgear cards use the tulip driver. during the install when you have to modprobe for your card “modprobe tulip” and you should be good to go.

i see alot of people saying they are newbies and trying it and getting frustrated. Gentoo is self proclaimed NOT a newbie distro. That said you can learn a schitt load doing an install and configure, and come out with a great machine in the end.

I have a few Gentoo boxen running, one of which is my file server at work running samba and netatalk to provide file and print sharing to about 30 macs and windoze machines. It has worked well for me – very well – but I don’t “–update world” EVER. Key is to update what you need and check what version you are updating too.

2002-05-15 4:59 pm

Source Mage (formerly Sorcerer) and Gentoo are both excellent.

Source Mage is easier to setup, install, maintain, and upgrade, and has auto-healing capabilities unmatched anywhere else (except Lunar, which is a Source Mage/Sorcerer fork).

But this comes at a price. There is (currently) no versioning support, which makes it difficult to have multiple versions of, say, autoconf and automake, or a library you want several versions of.

Gentoo’s weakness and strengths an almost exact mirror-image of Source Mage’s … where the one is weak, the other is strong.

Gentoo upgrades are tricky and risky (as the interview indicates), and you’d better know exactly what you’re doing before you upgrade a package. It is a little more complex (but worth it for the sandbox safety features and version support). It supports versions, making it easy to have multiple versions of KDE, autoconf/automake, and various libraries present as needed, and has pretty good support for precompiled binaries when needed (e.g. Blender).

Both of these projects are important … they have different but similar visions on how to accomplish something: leverage the power of source access with the simplicity of packaging a distro, so that normal folks using GNU/Linux can have the best of both. There is plenty of opportunity for both Source Mage and Gentoo to learn from one another and cross polinate while following their own visions (Gentoo desperately needs recursive rebuild capabilities, and the ability to audit dependencies in both directions [“I depend on that” as well as “these depend on me], while Source Mage really needs a sandbox scheme to make upgrades less intrusive (e.g. not clobbering a running X while upgrading a new one) and version support in its spell system so multiple library and application versions can coexist peacefully).

Whichever distro you use (and I use both), you will find the learning curve of dealing with a source based distro to be well worth it. Indeed, you’ll likely find you’re unable to go back to a binary distro, no matter how well packaged, once you’ve had a taste of the speed and stability a source based distro like Gentoo and Source Mage can give you.

2002-05-15 6:19 pm

What would be really cool is if Gentoo and Debian were combined. You could use Debian’s branches to give you the binaries and the base distribution, and Gentoo to give you all the bleeding edge stuff, so in addition to: apt, dselect or dpkg, which would try to install the binaries (if available) you would also have emerge-apt, emerge-dselect and emerge-dpkg which would try to build the source and use the installed debian binary packages if they were available.

Does anyone know how to compile Gentoo on Windows? How well does it work? Is GNOME or KDE ported?

2002-05-15 7:35 pm

I have been using Gentoo Linux for the past ten months as my primary work platform. It is *wonderful*.

At one point I got stuck on an installation problem. Daniel Robbins helped me fix the problem via IRC. I have never received better support!

These days, I run Gentoo Linux as a VMWare image hosted by Windows 2000 on a Dell 8000 Laptop — my job requires this particular configuration. It works very well.

I hope to install Gentoo natively now that I have another hard disk to play with. Looking forward to trying ACPI now that it’s included.

2002-05-15 8:10 pm

Eric, that is Gentoo the FILEMANAGER, it has nothing to do with the Gentoo, the Linux distribution.

2002-05-16 2:38 am

Matt,

The fact that you have to merge your /etc files by hand after an update is by design. That way, any changes you’ve made (system specific etc files are very common – fstab, X11/XF86Config, make.conf, etc) don;t get overwrittem automatically. Besides, 90% of the time, the new etc file is just like the old unedited version (hence, no changes needed). My recomendation is to back up ther file before editing, then compare the backed up version with the new version (ala ._cfg0000_***) to see if anything really changed. Hope this makes sense and doesn’t add confusion…

2002-05-16 2:44 am

“(Gentoo desperately needs recursive rebuild capabilities”

emerge –update world –emptytree ?

“and the ability to audit dependencies in both directions [“I depend on that””

emerge foo –emptytree –pretend

“as well as “these depend on me]”

grep -r foo /var/db/pkg/*/*/*.ebuild

2002-05-16 9:47 am

Gentoolkit has a nifty little script that summs the differences between the old and new config files.

if you only see lines eddited by yourself, you can emediatly delete the new uneditted version with a single keystroke.