Confessions of a Recovering NetBSD Zealot

The NetBSD Project has stagnated to the point of irrelevance.
It has gotten to the point that being associated with the project is often
more of a liability than an asset. I will attempt to explain how this
happened, what the current state of affairs is, and what needs to be done to
attempt to fix the situation.

Federico Biancuzzi interviewed Charles (who was affected by the recent
termination of some netbsd.org accounts) and discussed the evolution of the
project, funds management, problems with licenses and hardware documentation,
the link with vendors, past and current mistakes, and what we can learn from
the Linux development process.

Could you introduce yourself?

Charles M. Hannum: I'm one of the creators of the NetBSD Project, and served as its de facto technical lead for a long time. I was also involved in creating the NetBSD Foundation, and served as its president and chairman of the board. (Note: I was never the Foundation's secretary or treasurer.)

How did the NetBSD Project start?

Charles M. Hannum: The four people who started NetBSD were Chris Demetriou, Adam Glass, Theo de Raadt, and me. At the time, Chris and Adam were attending Berkeley and close to graduating. Theo was working for a living. I was doing other things. Chris is still sort of around, but doesn't really do anything; Adam went to work at Microsoft and is now lost to us; and we all know about Theo. There is some info in the NetBSD entry on Wikipedia.

I think the most striking difference between late 1992 and today is that
there really was no "web." The original software had been released, and some
of us had started using it for small (non-graphical!) websites, but it was
still very much a hobbyist thing. More people were getting "email accounts" of
various types, but penetration of the internet into homes was still quite low.
Even so, this was near the end of the NSFNet era, and there were increasing
problems with the backbone being overloaded. There were also some as-yet-undiscovered problems with TCP flow control which caused additional
performance problems. And lastly, nobody had started taking network security
seriously.

On the open source side, you could liken it to the Stone Age. The operating systems (primarily 386BSD 0.1 and Linux 0.12) were nothing to write home about; they were buggy and incomplete. There were no "desktop" packages or "office" suites of any significance. (I actually ported Applixware to NetBSD under contract, because a certain blue company wanted it for their thin clients that ran NetBSD.) Development of X was hampered by the dissolution of the X Consortium and a vacuum of leadership there. We discovered later that GCC development was also being hampered by mismanagement (since fixed).

What I'd like to stress is that there was no Dummies Guide to Starting
an Open Source Project. The term "open source" wasn't even being used yet, but that's beside the point. We were the first big collaborative project to use a version management system. (As examples, Linux used none, and most GNU software at best used "backup files" for version management.) There were no previous successes--or really even significant failures--to look at for guidance about how to organize the project. So we made it up.

I want to elaborate on the point about network security a little. Keep in
mind that, in this time frame, most people were still using
rlogin and unencrypted telnet. Buffer overflows were
rampant. I (and other people) had already started doing experiments with
subverting web sites; we all knew it was possible then, but nobody cared. It
was several years before most people saw the writing on the wall and started
clamping down access to central repositories and whatnot. Even today, most
software distributions on the net (and this includes NetBSD) are not
signed.

The primary reason for starting NetBSD at that time, ironically enough, is that there was a perceived lack of management in 386BSD. The actual 386BSD release only ran on a handful of systems, and was quite buggy. There was a rapidly growing community around it nonetheless, and many people had contributed patches. However, 386BSD's leader simply vanished. Nobody had any idea what he was doing, or whether he was even looking at the patches or working on another release. Eventually we decided that the only answer was to make a go of it ourselves--that's right, it all started with a fork.

Would you like to talk about the fork that originated OpenBSD?

Charles M. Hannum: No.

Since it came up in the /. thread, though, I would like to make one correction. It's widely claimed that I'm "the one" who ejected Theo from the NetBSD community. That is false. At that time in NetBSD's history, Chris G. Demetriou was playing the role of alpha male, and I wasn't even given a choice. I was certain it was going to bite us in the ass. I think the question for historians is not whether it did bite us in the ass, but how many times and how hard.

Why did you focus on portability?

Charles M. Hannum: At the very beginning, this was not actually a focus. It quickly became apparent that there were a large number of people interested in NetBSD (and open source OSes in general) who were currently on non-x86 platforms. Remember that this was still the 80486 era--PC processors weren't very good. Most "workstation"-class computers were based on something else--a myriad of Motorola 68k and 88k, SPARC, POWER, etc. On the HP9000/300 and SPARC platforms, we also had the advantage of access to code already written--though in both cases the integration was complex, and the code only supported a handful of systems at first.

I was also hanging out (and occasionally doing some work) at the Free Software Foundation, where there were a lot of HP9000/300s--running MORE/BSD, which had long since been abandoned. So I set out to get NetBSD running on these machines. Just getting a cross compiler working at the time was quite a bear, but it didn't take too long before these machines were all running NetBSD.

Yes, several of the earliest NetBSD development systems were owned by the Free Software Foundation.

With working 68k support in hand, we quickly spawned Amiga and Mac development groups. I then helped Theo get the SPARC support from LBL integrated. The whole thing snowballed.

Of course, the fact that you've ported the system to tons of machines does
not mean you have good portability. We were still in the days of "copy and
edit"--and so, for example, although the Amiga and Mac code [was] heavily cribbed from the HP9000/300 code, they were separate and had different features and bugs. Trying to fix bugs in this environment was making me crazy--and wasting a lot of time--and eventually I just started smashing the code together at high speed and sifting out the common parts. Other developers followed ("lead by example" works sometimes), and so this led into our global thinking about portability architecture, shared device drivers, etc. Oddly enough, Microsoft was working on similar ideas at the same time, in the development of Windows NT.

How was the relationship with hardware manufacturers at that time?

Charles M. Hannum: Terrible. We rarely got documentation from anyone. I actually wrote a lot of device driver code by guessing what the device was supposed to do. (Lots of previous experience reverse-engineering code helped there, I'm sure.) We had severe problems trying to deal with things like the "programmable" Adaptec SCSI controllers that became very popular; it was so bad that I was honestly talking about staging a sit-in at Adaptec HQ, and probably should have done it.

There were some notable exceptions, though. It took a while, but NCR, and later LSI, finally came around and dropped a heaping pile of (fairly good) documentation on me for their SCSI controllers. We did eventually get some material out of BusLogic, but only for their older "heavyweight" controllers (what we call "bha").

Intel, for its part, has been pretty good about putting CPU and chipset
documentation online for the last several years, which I applaud, but their
networking documentation (both wired and wireless) has been extremely poor.
The strangest part of the Intel story was when the i82559 manual became
restricted, even though it was substantially identical to the i82557 manual
[that] had been published in their networking databooks earlier. Most other
companies producing Ethernet controllers have been decent to us, except for
Broadcom and Marvell, which have [each] been a 100% loss. Wireless vendors have generally been a tremendous annoyance, generating excuses but no documentation--I think Atmel and Realtek are the only exceptions.

We got some scattered documentation from other companies--e.g. Ensoniq and Realtek--but it's sometimes been incomplete and very difficult to make sense of.

Fortunately we didn't have to deal with most of the PC video card circus ourselves; XFree86 and now X.org have taken care of that.

What type of relationship did you have with the license? Is this relationship changed during the time?

Charles M. Hannum: Most of us had a very strong distaste for the so-called "virus" clause in the GPL, and this is the primary reason we did not adopt it. There was also some thinking that CSRG (Berkeley) and the X Consortium had been successful with leaner, looser licenses, so why bother. In retrospect I think this was naive; if you look at the history, you'll see that neither CSRG nor the X Consortium were really successful in getting third parties to contribute back most of their changes--and so what we really got in both cases was a long list of derived but very different, and often incompatible, systems.

Linux has not been wholly successful in this either, and today there are myriad distributions which are subtly incompatible. However, they definitely did better.

If I were doing it again, I might very well switch to the LGPL. I'll just note that it didn't exist at the time.