NetBSD 2.0 Rendezvous

In December 2004, the NetBSD Project released the feature-rich NetBSD 2.0. Even after such a masterpiece, developers kept working on
improvements, new features, and new ports following the new development roadmap.
Federico Biancuzzi recently interviewed them to find out what they are working on and how they plan to promote their project in the near future.

NetBSD's goal is to port the OS to as many platforms as it can.
Which missing platforms would you like to support?

Christos Zoulas: We are currently working on IA64 and we
should have something to show soon. As far as other platforms go, it is quite
random. We need to have access to hardware or to a simulator, and find a
developer that has time to do the port. I'd personally like to see a port on
the Zaurus (should be easy because of our existing arm support), the IBM360
(which is a lot harder without help from IBM), and some of the big SPARC SMP
systems that we don't support (there has been some work on that).

Is anyone already working on new technologies like 10Gigabit
Networking, PCI-Express, and ExpressCard?

Hubert Feyrer: NetBSD already has support for these
devices: the dge(4) driver supports cards based on the Intel
i82597EX chipset, like the Intel PRO/10GbE LR. The i82597EX chipset supports
IPv4/TCP/UDP checksumming in hardware, as well as TCP Segmentation Offloading
(TSO). The driver does currently only support the hardware checksumming
features, which can be enabled with the ifconfig(8) command. The
driver also makes use of the link flags link0 and
link1 to set the PCIX burst size; see the dge(4) manpage for more
information.

Support for PCI-Express is already available in NetBSD's machine independent
PCI bus interface, which is used for all hardware platforms NetBSD runs on,
including, but not limited to, Intel- and AMD-based PCs, Opterons, Sun
UItraSPARCs, and Apple PowerPC machines.

An example of NetBSD's performant support of this hardware can be seen in
the Internet2 Land Speed Record achieved by SUNET. Using two off-the-shelf PCs
with the above-mentioned cards made it possible to transfer data in a
production network between Lulea, Sweden, and San Jose, U.S. at an average rate
of 4.3GBit/s over the duration of an hour, with peak throughput at about
8GBit/s for a single stream. See SUNET
Internet2 Land Speed Record: 124.935Pbmps (single stream) and SUNET Internet2 Land Speed Record: 122.367Pbmps (multiple stream) for more information, including details on the
network infrastructure and endpoints and their configuration.

FreeBSD 5.3 and DragonFlyBSD 1.0 provided a multi-threaded network
stack, and now their developers are working on other sub-systems too. Do you
plan to follow the same path?

Christos Zoulas: We are definitely going to multi-thread
the network stack very soon. This is necessary in order to achieve wire-speed
performance on gigabit+ networks. I don't think that this will make it for 3.0,
because it is scheduled to be branched in February. This release will
incorporate significant fixes to our NewReno code, SACK, and TCP Westwood+
system.

FreeBSD 5 introduced the new scheduler called ULE, and Linux 2.6
included a new O(1) scheduler. What about NetBSD?

Luke Mewburn: No specific plans at this time. We will
observe FreeBSD's experiences with ULE and consider importing it if it is
beneficial in the long run.

Hubert Feyrer: As far as I understand, there is a certain
desire to have pluggable scheduling modules in NetBSD to accommodate different
schedulers than the classic round-robin scheduler NetBSD uses right now. If and
when this happens to materialize is not yet known due to limited capacities
available in volunteer projects like NetBSD.

FreeBSD developed a framework to support binary drivers for MS
Windows. Is this something interesting for NetBSD, too?

Hubert Feyrer: It is interesting to NetBSD, of course, as it
enables using proprietary, closed-source/binary-only drivers at least on one
platform, where no driver at all would be available otherwise. Of course, we
much prefer proper drivers based on NetBSD's driver framework, which requires
vendors to release documentation along with the hardware they sell. That way,
people would benefit not only on one hardware platform, but any of their own
choice.

As it currently stands, it's nice to have Atheros, Nvidia and NTFS drivers
available that way on PCs, but leaving out all the fine Mac and UltraSPARC
hardware seems suboptimal from NetBSD's point of view.

OpenBSD started
an effort to include in the main code base drivers for E1/T1 or T3/E3 to be
a valid and cheap alternative to Cisco products. Is this be something
interesting for NetBSD, which is known to be easily portable to embedded
devices?

Luke Mewburn: Sure. There is a lot of "cross pollination"
of drivers and subsystems between the BSD projects, which is good.

I've read some epicthreads
about the inclusion of PF in the main NetBSD code base. Now that Itojun has imported it, how will
PF and IPF interact?

Luke Mewburn: There are some issues remaining with the PF
integration, especially in terms of the PF + ALTQ interactions.

A longer-term goal is to separate out a packet classification so that the
code can be shared between systems such as IPF, PF, and ALTQ without
unnecessary replication.

James Chacon: PF is being included as another packet
filtering choice for users to have available to them. In the end it's expected
users can pick/chose whichever software best meets their needs here, so being
able to offer both is a big win.

Christos Zoulas: PF and IPF are competing products, each
one having its strengths are shortcomings. NetBSD will support both, and may
the best of breed win.

Is anyone working on virtualization extensions to have multiple
independent systems running at the same time?

Hubert Feyrer: Yes, in two ways: on a kernel level, there
is NetBSD/xen, which acts as a virtual machine that can run several concurrent
operating systems, with NetBSD being one of them (besides Linux and Windows).
NetBSD was especially prepared to run as "Domain 0"; i.e., as the first machine
on top of the Xem VM.

On a userland level, work is underway to do something like Usermode Linux on
steroids: not only are system calls, etc. trapped, but also CPU instructions,
which allows running non-native binaries. An example implementation of
NetBSD/arm32 binaries running pkgsrc bulk builds on a NetBSD/i386 system was
demonstrated at the pkgsrcCon in May 2004 in Vienna, and it was very impressive
to see the system compile packages in near-realtime.

Do you have any plans to include or develop a kernel framework for
clustering like DragonFlyBSD?

Christos Zoulas: We would like to eventually, but this is a
longer-term goal for us. Improving the filesystems and realtime support are
higher priority for us, and easier to achieve.

Do you plan to use systrace to alter the common Unix permissions
model without introducing MAC extensions like FreeBSD 5 did?

Niels Provos: Well, that would be one application that you
could use it for. I think about it as a tool that should be easy to understand and
deploy. In many cases, MAC systems are very complex and very difficult to
understand. systrace is not perfect, but provides some easy-to-deploy security
enhancements. I think that it is going to be important to make it work well for
the average user; e.g., many users download untrusted software and run it.
systrace could prevent any malicious code that might be hidden in there
causing damage to the system.

Do you have any plan to replace the old plain-text logs format with
something like Bruce Schneier's Cryptographic
Support for Secure Logs? Tampering with logs is pretty easy, and this has
gone unaddressed for years.

Christos Zoulas: We have not considered this yet. While it
sounds like a good idea, it has the drawback that the log files are not simple
text anymore. I personally think that it is better to centralize log collecting
in a secure machine, because cryptographically protecting the logs only helps
you to verify that the logs have been tampered with. It does not help you to
recover the information that has been potentially lost. We do have append-only
files right now, and this seems to serve us well so far.

Do you plan to use Verified Exec to distribute binary
updates?

Brett Lymn: I really cannot speak for the project. It would
be nice to have as an option, but verified exec is not really something that I
would recommend for a general-purpose workstation because it can get really
annoying very fast--it is more aimed at providing an extra level of protection
for machines like firewalls, web servers, and the like. Distributing the
fingerprint file is a tricky thing to do; issues like making sure it has not
been modified in transit, who is allowed to generate the file, and where the
file is generated are all difficult questions that need to be solved before an
official fingerprint file can be produced. At the moment, I have no firm plans
to push this now but I may ask the project about doing this later.

Christos Zoulas: No, we are going to be using the
standard hash signature support for that (MD5, SHA1 etc). Note though that we
have /etc/mtree/set.* (per-set mtree specfiles) that contain a SHA1
hash suitable for more easily generating veriexec hash lists and/or validating
a system.

What is the status of package views support?

Christos Zoulas: Some packages have been converted to use
views, but not all. Package views are targeted for the pkgsrc-2005Q2 release,
which will take place in June 2005, but may be brought forward if possible.
There has also been a lot of pkgsrc work in porting to other platforms--the
list of currently supported platforms looks like:

AIX

BSDOS

Darwin

FreeBSD

IRIX

Interix

Linux

NetBSD

OpenBSD

SunOS

UnixWare

There's also been a lot of work in cutting the number of broken packages via
regular bulk builds on a number of platforms, by a strict monitoring of any
new features in the infrastructure, and by rapid fixing of anything that is
broken.

Alistair Crooks: The infrastructure for package views is in
place, and a number of packages (maybe 30-40 percent) have been converted to be able
to take advantage of package views. We are targeting Q1/Q2 2005 for having
all packages converted to be able to take advantage of package views, although
we'll retain the existing "overwrite" functionality (it is desirable in a
number of situations, such as on embedded machines).

Subversion 1.1 has been released. Do you have any plan or desire to
adopt it instead of CVS?

Hubert Feyrer: Not yet. There is no experience with a
repository as big as the NetBSD one, and the import routines from CVS to
Subversion need a lot more work before all the information kept--and used!--in the NetBSD CVS repository can be transferred into Subversion easily.

Luke Mewburn: Not at this time. We need to further analyze
the long-term benefits and disadvantages in migrating our version control
system from CVS to another technology. It may be that Subversion doesn't
provide us with enough benefits over CVS to be worth the hassle of converting.
There are other open source version control systems available as well that
advertise as being "CVS replacements."