Unfilling my inbox: NetBSD news from the past few weeks - ACPI, NUMA, Xen, and more
Herre are some more things that I've caught in my inbox for too long,
and I'm finally finding some time
to sum them up here:

NetBSD's "let's move kernel parts to the userland" RUMP
project is still under heavy development, and in order
to make testing of compatibility after kernel changes easier,
a new command "rumptest" was added to build.sh:
``Basically you say:

./build.sh ${yourargs} tools ; ./build.sh ${yourargs} rumptest

Where yourargs are what have you, e.g. '-U -u -o -O /objs'.

The latter builds only the rump kernel libs and uses some ld+awk magic
to figure out if things go right or not. This is to avoid having to
install headers and build libs (which is too slow since a full build is
too slow). The magic is not a substitute for a full build, but it is
n+1 times faster and works probably 99.9% of the time.

The scheme uses a number of predefined component sets
(e.g. tmpfs+vfs+rumpkern) to test linkage. They are currently listed
in build.sh. This area probably needs some work in the future. It would
be nice to autogenerate the combinations somehow.

According to
Wikipedia,
``Non-Uniform Memory Access or Non-Uniform Memory Architecture (NUMA) is a computer memory design used in multiprocessors, where the memory access time depends on the memory location relative to a processor. Under NUMA, a processor can access its own local memory faster than non-local memory, that is, memory local to another processor or memory shared between processors.''

Staying with ACPI and Christoph Egger, he found that even
though the ACPI spec defines an ACPI device for fans,
BIOS vendors and OEMs do their own thing.
To accommodate things like the fan sensor found in
the ACPI Thermal Zone in his HP Pavillion DV9700 laptop
he has
proposed a driver
to extend the acpitz(4) driver with fan information.
That way, envstat(8) can be used to display the ran's
RPMs:

The documentation covers how to enable the
Direct Rendering Manager (DRI), setting up and configuring
Modular X.org, assuring that everything's in place, and
how to get
Compitz going. Mmm, wobbly windows at last! :-)

While we're talking funky desktop stuff: Marc Balmer has
submitted
a patch to get touchpanel support for ums(4).
ums(4) is for USB mice, and in contrast to mice, touch panels need
to deal with absolute numbers, not relative numbers.

Back to the guts of the kernel, another patch suggested
by Christop Egger was for
adding x2apic. What is x2apic?
X2APIC is
``an Intel-only feature but can also be found
in virtual environments with support for CPU apic id's > 0xff.

Last one for today: Michal Gladecki,
Editor-in-Chief of BSD Magazine
writes:
``We are happy to announce that BSD Magazine is transforming into a free monthly online publication. The online version of BSD Magazine will stay in the same quality and form. It will look like the BSD magazine one is familiar and comfortable with. Please sign up to our newsletter at www.bsdmag.org and get every issue straight to your inbox. Also, you can now download any of the previous issues from our website. The first online issue -- 2/2010 -- is coming out in February. Please spread the word about BSD Magazine. ''
Click!

So much for today. I still have a bunch of news items
in my inbox for next time, but let's call it
good for today.

Unrelated, I've been playing with git a bit over the
past few days, and wile I have a number of questions building up
(which will be subject to tech-repository or so), what I
can say today is that the speed of "git pull" with
NetBSD's git repository and my 1MBit DSL line reminds me
a lot of the times when I used SUP with my 56k modem
- it took forever, too. :-(

With Webfwlog you can design reports to use on your logged data in whatever configuration you desire. Included are example reports as a starting point. You can sort a report with a single click, "drill-down" on the reports all the way to the packet level, and save your reports for later use. You can also create a link directly to any saved report.''

Git copies of cvs modules available -
spz@ writes that ``htdocs, othersrc, pkgsrc, src and xsrc are now available as
(bare) git repository copies at
http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.
These repositories are currently updated every 30 minutes.

They are not authoritative and are not meant to replace cvs,
but if you want to track your development that is not yet fit
for committing into cvs head or sending in as a PR, and want to
use git for that, they may serve as a convenient starting point
or reference.''

Support for writing file systems on sparse disk images -
File system hacker Antti Kantee writes:
``Yesterday I wanted to write to a file system which was on a sparse
disk image. Normally I would have just used a rump mount, but since the
file system was not at offset = 0 in the file, this failed. vnd wasn't
helpful either: it allowed me to mount the image and then I got weird
errors when writing.

I just finished adding disklabel support to the various rump_fs utilities.
Due to them using the option parsing code of the real mount_fs utilities,
I decided to signal the label number with a magic parameter at the end
of the device (yes, this really simplifies things currently).

Throttling the CPU to reduce fan noise
On netbsd-users, Cem Kayali has
posted a script to adjust the CPU frequency according to the system temperature,
and thus reduce fan noise. This is different from pkgsrc's estd,
which just adjusts the CPU speed according to the system load, so
it's quite the contrary. Cem uses the envstat(4) framework to
determine CPU temperature, and then slows the CPU down with sysctl(1).

An alternative approach was
suggested by Jeremy Reed,
which hints at the powerd(8) script hooks that can be set to
act on corresponting ACPI Thermal Zone (acpitz) events.
See
Jeremy Reed's blog posting
for a few more ideas, plus a hint that this may become
documented even better in the future.

source-changes catchup mid-July to early September 2008 (Updated)
Welcome to yet another catch-up of NetBSD source-changes mailing list,
this time from mid-July to early September 2008. Besides FFS having
journaling now (yai! first in BSD-land, ever! :-), here's what's new
and/or exciting:

In order to re-initialize x86 machines' video/VGA state after
suspend and resume, some BIOS functions can be used. This needs
to be done in real mode(?), which is a bit hard to do from an
operating system kernel that runs in protected mode. To help
doing so, a x86 CPU emulator was added to NetBSD some time ago,
to help run VGA bios for ACPI resume. Now Joerg has added a
sysctl that does just this, assuming your kernel has the VGA_POST
options -- set machdep.acpi_vbios_reset=2

Inside the kernel, data sent/received through the network stack
is stored in chains of
mbufs. So far, the mbufs were also used to store socket
options, i.e. data describing further how the sending/receiving
is done. This was split out into a separate struct sockopt by Ian
'plunky' Hibbert now. For more information, see sockopt(9).

Hans 'woodstock' Rosenfeld has added a new accalerated driver for
SPX graphics boards found in some VAXstations, which replaces the
old and broken lcspx driver. The work is based on work by Blaz
Antonic.

The simonb-wapbl branch was merged: ``Add Wasabi System's WAPBL (Write Ahead Physical Block Logging)
journaling code. Originally written by Darrin B. Jewell while
at Wasabi and updated to -current by Antti Kantee, Andy Doran,
Greg Oster and Simon Burge.''
This makes NetBSD the first second (see update below)
BSD operating system that has a working
file system with journaling (not counting LFS, which again and
again has issues). Mmm, no more fsck! :-)
See my other
posts for more on journaling / wapbl.

Update:
James Mansion wrote me to that NetBSD's not the first BSD to
have journaling, and I think he's right:
DragonflyBSD's HAMMER file system apparently offers similar
functionality: ``HAMMER implement an instant-mount capability and will recover information
on a cluster-by-cluster basis as it is being accessed.''

Accept filters were ported from FreeBSD by Coyote Point Systems,
and integrated into NetBSD by Thor Lancelot Simon. What are
accept filters? According to the accept_filter(9)
manpage, they ``allow an application to request that the
kernel pre-process incoming connections.'' Pre-defined
filters are available with
accf_data(9) and
accf_http(9). The latter makes sure that the
application's accept(2) call only sees the connection if there's
a valid HTTP header, moving parts of the parsing from userland
(httpd) to the kernel.

Work is underway for crossbuilds of modular X.org. This is done
via src/external/mit/xorg, which needs xsrc/external/mit. The
results will be installed in /usr/X11R7(!). (XXX Where can I find
more about this?)

Gregory McGarry is working to get the tree compiled with PCC
instead of GCC. This is still ongoing.

nvi was updated from version 1.79 to 1.81. The most important
part of this update is that internationalization is now handled
by default.

Following a bigger masterplan, new 3rd party software packages
are now imported into src/external/${license}, which will replace
src/dist, src/crypto/dist and src/gnu/dist in the long
run. Packages will be moved on upgrades only, existing packages
are not being moved just for the sake of moving them.

Adam Hamsik is working on getting Logical Volume Management (LVM)
going in NetBSD. He has adapted Linux' "device mapper"
kernel-interface as part of his Google Summer-of-Code project,
and with the help of the (GPL'd) Linux tools, things are looking
pretty good. More on this in a separate post. This work is
currently happening on the haad-dm branch.

In the context of his work on UDF, Reinoud has added routines for
speeding up directory handling by using hash gables. Lookup of
files was O(n*n) and is now O(1) even for file creation.
See my
other blog posting for details and impressive numbers.

Perry Metzger is working to make binary builds identical. This is
useful for binary diffs between releases/builds, e.g. when
providing binary patches for updates and security fixes. Areas
where this had an impact on are C++ programs and various
bootloaders (which had a builder, build date, etc. in it so
far).

EHCI (USB) can now do high speed isochronous support. This was
developed by Jeremy Morse as part of his Google
Summer-of-Code "dvb" project this year, it is useful for fast
transfer of data that comes in steady streams, e.g. from video
cards.

fsck_ffs(8) now has options -x and -X (just like dump) that
create a file system snapshot via fss(4), and then operates on
the snapshot. This allows "fsck_ffs -n" to work on a snapshot of
a read/write mounted file system, and avoid errors related to
file system activity. Can be made permanent for the nightly
script by setting run_fsck_flags="-X" in /etc/daily.conf.
This was brought to you by our Xen-hacker Manuel Bouyer. :-)

So much for this time. Many of the above projects are
work-in-progress, and we can look forward for further news on them
next time. Stay tuned!

Catching up on source-changes (Feb 21st - Mar 1st)
Mark Kirby's CVS digest is still down. I'd love for someone to continue this service - please
contact Mark and/or me for details! (I don't really have time to do this,
but will try to do so. Help is definitely appreciated!!!)

Catching up on NetBSD's
source-changes mailing list from
Feb 21st 2008 to Mar 1st 2008, here's what people not following
things closely may find interesting:

Chris Gilbert made process on the chris-arm-intr-rework
branch, which aims at reworking the ARM interrupt code to provide
a shared set of interrupt routines that can be used across
all ARM based machines.

The PowerPC routines for bcopy, memcmp, memcpy and memmove were
updated to use tuned versions that were written by IBM and released
under a 3-clause BSD license as part of the perflib project, see
http://sourceforge.net/projects/ppcperflib/.

Keiichi Shima has imported the Mobile IPv6 code developed by
the SHISA project. The
work can be found on the "keiichi-mipv6" branch.

Matt fleming has started work on a device file system that supports
dynamic device nodes, which is of relevance when loading kernel
modules for device drivers, so you don't have to run /dev/MAKEDEV
manually. The work is on the "mjf-devfs2" branch.

Nick Hudson is currently working on syncing NetBSD's WiFi (net80211)
code with FreeBSD, so that these two operating systems (and possibly
others) can share a common codebase. The work currently affects
only the kernel, but this may extend to the userland eventually.

The glob(3) wildcard extension routines got a new options
GLOB_NO_DOTDIRS added, which hides . and .. even if the pattern
would otherwise include them.

Juan Romero Pardines has ported the mfi(4) driver from OpenBSD. The
driver supports disk controllers with PowerPC IOSs such as the LSI
SAS1078 and the Dell PERC6.

The boot-only ISOs built for releases included several kernels
(with ACPI, without ACPI) for some time now. Stephen Borrill has
added a menu to make kernel selection easier. Mmm, user friendliness!

The machine-independent versions of strcspn(3), strpbrk(3) and
strpspn(3) were improved from O(mn) to O(n+m) based on ideas from
DragonFlyBSD.

Power management hooks for suspend, shutdown and resume were
added to many device drivers, to support the new power management
framework by Jared McNeill.

A whole lot of changes were pulled up from NetBSD's development
branch into the release branches for NetBSD 3.x and 4.x.

So much for this round of updates. A lot of work was not done on the
main trunk but on some branches. People interested to learn what branches
exist in NetBSD's source tree can check out src/doc/BRANCHES to do so.
Enjoy!

Catching up: netbsd.se web design contest, CuWIN, cobalt restore CD, ...
Ok, some more busy days have passed, and I feel like I should post
about things that have happened. Sorry for not being more upto
date. So, what happened in NetBSD land?

Andre Pettersson sent mail about a
web design contest for www.NetBSD.se.
Besides eternal fame, the winner will get $150US. Check the mail
for some requirements and where to send submissions!

Mike Volokhov pointed me at CUWiN, the the Champaign-Urbana Community
Wireless Network, which is
``a world-renowned coalition of wireless developers and
community volunteers committed to providing low-cost,
do-it-yourself, community-controlled alternatives to contemporary
broadband models.''