The original motivation of Aboriginal
Linux was that back around 2002 Knoppix was the only Linux distro
paying attention to the desktop, and Knoppix's 700 megabyte live CD
included the same 100 megabytes of packages Linux From Scratch built, which
provided a roughly equivalent command line experience as the 1.7 megabytes
tomsrtbt which was based on busybox
and uClibc. If I could free up an extra 100 megs of space on Knoppix's boot
CD they'd work wonders with it, so I started work trying to build a fully
functional Linux From Scratch with all the gnu bloatware replaced with
busybox and uClibc.

The quest for the smallest Linux system capable of rebuilding itself
under itself (and building the original Linux From Scratch under the result
as a proof of equivalent functionality) yielded
a system built from 7 packages: linux, busybox, uClibc, gcc, binutils,
make, and bash. With a build control image
that build Linux From Scratch, the project had met its initial design
goals and had its 1.0 release way back in 2010.

Along the way I did so much work on BusyBox I wound up maintaining it,
and cleaning up after Busybox' "hall of shame" made me care more about
licensing issues than most people, which is why when GPlv3 happened
I said no. And I similarly
refused to ever ship a GPLv3 package in Aboriginal Linux, which meant I
had to maintain the last GPLv2 release of gcc and binutils, patching my
way around breakage every time a new kernel came out that depended on a
new feature.

I managed this for about a decade, but after Linux 4.3
the next two kernel releases broke four different architectures,
which was more than I had spare time to deal with just then.
Strategically, switching to clang/llvm/lld would be a better investment
than more work on the old gplv2 toolchain, but it was missing support for
most architectures (including superh, which I work
on elsewhere. Once upon a time I'd hoped to move to
qcc but making Android self-hosting became a more
important use of time. This left Aboriginal Linux with no way forward
that wasn't either a lot of work to produce any result, or a waste of
time in the long run.

So the project lay dormant until the musl-libc maintainer created his own
toolchain builder,
and a new (simpler) design became possible. With a little persuasion,
he built not only cross compilers but portable _native_ compilers for the
supported targets, which was the thing I'd never managed to get out of
any external project before. (The closest was Debian, which dynamically
links its compilers against the host's glibc, and doesn't support all
targets.)

The new design provides a
better test harness for toybox, and better serves the goal of making
Android self-hosting. The Android guys are working on their own toolchains based
on LLVM, and Rich has plans to host binaries once I've tested all
the targets and shown that what we're building is good enough. (I refuse
to distribute them, but others are free to take that risk.)

Given a choice between shipping GPLv3 binaries and ending Aboriginal
Linux, it was no contest. And thus the project ended.

Dreamhost deleted a big chunk of history out of the mailing list web archive
again. Last year it was just a few weeks, this time they retroactively zapped
11 months (almost all of 2015).
I provided some details
in another affected project,
but given that lasttime
they never filled in the previous gap in the archive, I don't hold out much
hope. If history is a guide, they'll keep apologizing as long as I keep poking
them about it, but nothing will ever get fixed. (Some of the vanished posts
are in archive.org.)

Due to the musl conversion, new build control
images are necessary to provide dropbear, strace, and linux from scratch.
(Both strace-4.9 and Linux From Scratch 6.8 fail to build under musl for
basically stupid reasons, in the case of LFS more than one package actually
has an #if/else staircase for every known libc type ending in an #error if
it doesn't recognize it. Since musl doesn't like and pretend to be glibc
like uClibc does, I need to upgrade package versions...) Those aren't
uploaded yet, but should be available soon.

The system-image.sh build is now called every time by build.sh and
handles its own dependencies: it rebuilds the rootfs, native compiler,
and kernel images as necessary. (This means if you tweak the root
filesystem contents, system-image.sh doesn't have to rebuild the kernel each
time unless you're using SYSIMAGE_TYPE=rootfs instead of the default
SYSIMAGE_TYPE=cpio so faster build and test cycles.)

Several new config variables:

MY_PATCH_DIR - lets you specify an additional patch
directory applied after sources/patches (in this case with one subdirectory
per package instead of the package name in the prefix, so
sources/patches/linux-thingy.patch would be patchdir/linux/thingy.patch
instead).

MY_ROOT_OVERLAY - replaces SIMPLE_ROOT_OVERLAY, still a directory
of files to cp -a into root-filesystem.

MY_CROSS_PATH and MY_CROSS_PREFIX - specify an external
cross compiler. Providing a native compiler in that case (via MY_ROOT_OVERLAY)
is your problem, native-compiler.sh generally won't work without a corresponding
cross compiler built from the same source.

You can now specify EXTRACT_ALL=force to populate the page cache
(extract tarball and apply patches) even when the packages directory
has a repository version of a package. (If you can't think of why
you'd need to do that, you probably don't.)

The miniconfig.sh creation script got moved from soures/toys into
more/, and the mini.config used to build the kernel tarball is now
saved in the system image tarball.

November 5, 2015

Aboriginal Linux 1.4.3
is out, based on hg 1807, using
Linux 4.1, toybox-0.6.1, musl-1.1.12, and busybox-1.24.1.

Our first nommu target, sh2eb, builds a system-image that boots on the
Numato Mimas v2 board described on the jcore
processor site (rather than under qemu).
Toybox defconfig now builds nommu, we upgraded busybox versions to
get some nommu fixes in hush, and there were several toolchain fixes
since last time.

Multiple users requested bzip2 compression-side back (even though you
can build it natively), and we added strndup() from posix-2008 to the last-ever
release of uClibc (which we're migrating off of in favor of musl-libc,
but our old Linux From Scratch 6.8 build control
image refuses to build on a libc it doesn't specifically recognize
so we need to upgrade the version of that to have a better test environment.

We added an /etc/os-release file which is what Debian now recommends
for OS version identification now that they've
given up on the Linux Foundation's
"Linux Standard Base" bureaucracy. Also the initramfs should have a /bin/sh
symlink pointing to hush (which is replaced with a link to bash by the
dev-environment.sh init script, but if you run-emulator.sh without the
native-compiler overlay it's nice to have).

September 13, 2015

Aboriginal Linux 1.4.2
is out, based on hg 1786, using
Linux 4.0, toybox-0.6.0, and musl-1.1.11.

Haven't updated busybox in a while because we're slowly replacing it with
toybox. The list of busybox commands still in use is:

The new "sh2eb" and "sh2elf" targets produce working cross compilers
for the jcore j2 open processor, and provide
a start for general nommu support. Userspace is not yet cooperating, mostly
because I'm only halfway done cleaning out MMU assumptions in toybox, but
I hope to hae that working next release.
(This release switched from ash to hush as the /bin/sh busybox provides
because hush works on nommu and ash doesn't.)

The top level build.sh script now knows that if a target doesn't specify a
KERNEL_PATH (where to grab the kernel binary out of the linux build), don't
bother building anything but the cross compilers. This lets sh2elf stop
early but still participate in buildall.

Switched the x86-64 target to build with musl-libc instead of uClibc
by default. The x86-64 target broke when we switched the uClibc config
from pthreads to NPTL in January, but only certain uses of it: the
more/chroot-splice.sh build still works fine. The symptom is that
distcc's attempts to call out to the cross compiler from within qemu
hang, due to some NPTL bug in uClibc that's not worth fixing because
uClibc is going away. (I'm aware of uClibc-ng, I'm also aware of
ecommstation's attempts to keep OS/2 alive.) This problem is further
compounded by QEMU's failure to emulate x86-64 floating point accurately,
so the perl
build breaks in the lfs-bootstrap build
control image. Since this is already screwed up beyond easy
fixing (other than reverting back to pthreads), I just switched it to
musl because putting effort in making musl work isn't wasted the way
it would be in uClibc. (The blocking issue with musl is that the
old Linux From Scratch 6.8 build is full of packages with #if glibc/
#else if uClibc staircases that #error out at the end if they can't
identify your C library. I need to update to a current LFS build
in which those packages have hopefully been fixed.)

Also updated e2fsprogs build in host-tools.sh to a newer version.
(It's not used by the build itself, but the dev-environment.sh script
uses it make writeable scratch space images for qemu native builds.)
This is because gcc 5.2 defaults to building in c99 mode, which breaks
the older e2fsprogs, and that may start showing up in people's host
linux distributions at some point.

May 6, 2015

It's basically just the kernel version upgrade, still a release behind
but no longer _two_ releases behind. Something broke the x86-64
lfs-bootstrap build under qemu (floating point rounding error in the perl
build, which dies with ""Unable to find a perl 2.36242958346314e-310"),
so I did a chroot version for now.

March 11, 2015

Aboriginal Linux 1.4.0
is out, based on hg 1748, using
Linux 3.18 and toybox 0.5.2.

The simple root filesystem now lives in initmpfs,
with the native compiler in squashfs on /dev/hda and spliced
into the $PATH at runtime by the init script. (This also provides an example
of how to do that on other systems.)

You'll need an updated build control image to
automatically build things like Linux From Scratch which want to setup a
chroot. (The old "should we copy our root filesystem into a chroot before
modifying it" test was "is / writeable", the new test is "is /home bigger
than /". The new root filesystem is writeable, but only a few dozen
megabytes large.)

This release has switched to gzip tarballs instead of bzip2, due to the
latter slowly becoming obsolete (as evidenced by kernel.org dropping use
of the file format). The busybox implementation of the gzip compressor
in insanely slow (slower than bzip2!) but it works and is scheduled for
replacement by toybox.

Toybox is now handling sed, which means the remaining busybox commands
which toybox still needs to replace are:

Of these 31 commands, toybox implements three (bunzip2, bzcat, and zcat)
which busybox can't use due to busybox "tar" requiring then to be built in.
Toybox' "pending" directory already has at least partial implementations of dd,
gunzip, pgrep, route, test, diff, ftpget, gzip, ping, sh, tr, xzcat,
expr, ftpput, pkill, fdisk, ps, and tar. This leaves ftpd, wget, less,
sha512sum, awk, getent, man, and vi still to be implemented.

December 27, 2014

Aboriginal Linux 1.3.0
is out, based on hg 1715, using
Linux 3.17 and toybox 0.5.1.

Patched out the bash function definition exports for that heartburn thing,
fixed several hiccups in the RECORD_COMMANDS functionality (logging what
the build actualy does while compiling stuff), broke x86-64 by enabling
TLS (doesn't quite work reliably, it seems), worked around a busybox bug
where sed -i on a nonexistent file would create it with random permissions
(toybox sed, coming soon), wrestled with first nommu platform (not ready yet)...

The usual. Still catching up to the kernel releases.

September 26, 2014

Aboriginal Linux 1.2.9
is out, based on hg 1688, using
Linux 3.16 and toybox 0.4.9.

Just the two package upgrades this time, I was still a kernel release
behind and I didn't upgrade toybox to the release back in June. New version
of each coming up in a week or so, so another catch-up release.

This also goes back to using busybox's cpio for now, because the linux
from scratch build uses the as yet unimplemented cpio -s (in toybox source
control, but not in the 0.4.9 release).

September 9, 2014

Aboriginal Linux 1.2.8
is out, based on hg 1684, using Linux
3.15 and a new ccwrap supporting musl-libc.

This is an interim release. An upgrade to linux 3.16 and a new toybox
release are pending.

The big news is the new option to build musl-libc images with the new config
knob MUSL=1, although the default is still uClibc for the moment because
everything needs to be retested thoroughly, and not all targets are supported
by musl yet.

Making even this much work required a from-scratch rewrite of ccwrap.c
(which started life as the old uClibc compiler wrapper, before they gave up
and started buildroot, and is thus full of uClibc-specific assumptions).
This rewrite caused regressions galore, and just making uClibc work again
with the new wrapper (for regression testing purposes) took a lot longer
than expected. It's back to building Linux From Scratch 6.8 with uClibc
again: next up is to build those same packages with musl. (And upgrade toybox,
and to the 3.16 kernel...)

The new musl targets are still a bit... wobbly. Static linking against
musl works, dynamic needs fixing. (It looks like -Bsymbolic-functions
in the linker isn't always working yet.)

Other bugfixes: patch toybox not to build break when uClibc does something
stupid with iconv.h. (New toybox release has a proper fix, but that's for next
time.) SYSIMAGE_TYPE=initramfs is using the right image name
again. The usr/src directory now has a hello.c (the README said it did,
but only the C++ and threaded versions were actually there).

The sh4 kernel config is no longer forcing CONFIG_EXPERT (that
patch should probably go upstream at some point, but after the number
of years perl removal took it's hard to muster any enthusiasm for engaging
with linux-kernel. Oh, and the qemu commit that broke signal handling
in qemu 2.x was qemu git 02c4bdf1d2ca, I pinged qemu-devel about that but in
case you want to revert locally and rebuild your qemu-system-sh4... It
works fine for automated native builds either way, but interactive use
is inconvenient when ctrl-c kills the _emulator_ instead of passing through
to the shell instance.)

Speaking of qemu upgrades, two kernel patches went away: the sh4 one to
not touch register 18 (qemu no longer exits over that), and qemu's powerpc
board can now emulate 4 drives hooked up to the primary disk controller, so we
don't need to use the cdrom controller to get an hdc, which means native
builds don't need us to patch Linux's insane scsi device probing ("throw all
the devices in a big pot and give it a stir, then grab one at random to be
sda") to align qemu's idea of -hda, -hdb, and -hdc with what Linux hallucinates
them to be. (Now there's only one controller and the Linux scsi guys are still
working on screwing up device ordering _within_ a single controller.)

Bobby Bingham sent in a patch to stop yet another hardwired path
stupidity where gcc was still reaching around ccwrap.c to screw up.
(Rewriting the path to start with --nostdinc --nostdlib isn't quite _enough_
to get gcc to stop imagining paths to check; you have to patch out bits
o the C code that try to grab random stuff, because gnu.)

Also, more/record-commands.sh infrastructure got upgraded to record the
second half of the host-tools build (rebuilding the wrapper directory each
time we add something to build/host, but in a way that _doesn't_ rebuild
the directory if it already exists to avoid interfering with multiple
parallel builds ala FORK=1 more/build.sh).

May 3, 2014

Aboriginal Linux 1.2.7
is out, based on hg 1649, using
Linux 3.14, and toybox 0.4.8.

Overdue maintenance release to catch up to the kernel. Yes, we skipped a
kernel release. And I meant to upgrade to busybox 1.22.1
but thought I already had because the last digit matched. And the musl
switchover isn't finished yet. Yeah, my todo list still runneth over, and
most of it's going to toybox.

At least the native stuff is building a new dropbear version. (Remember:
the NSA didn't put the goto fail exploit in iOS, or the other goto fail
exploit in gnutls, or the heartbleed bug into openssl, and I can't WAIT to
see what they didn't do to dropbear.)

Tested under qemu 2.0, and in fact powerpc requires it because /dev/hdc
moved. (Don't ask me why qemu-system-sh4 kills the emulator
when you hit ctrl-c, it's been doing that for a while. They left the terminal
in cooked mode and didn't intercept signals and I should probably poke them
about that. It's on the todo list...)

November 19, 2013

Aboriginal Linux 1.2.6
is out, based on hg 1638, using
Linux 3.12 and toybox 0.4.7.

There's a bugfix for hosts using patch 2.7.x, and creating sparse disk
images for the native builds uses the "truncate" command now rather than dd.
Gunter Roeck fixed a build regression if you want to build ext2 system images,
and Patrick Lauer made the smoketest timeout configurable (and extended the
default to 3 minutes).

(I'm way behind on email and distracted by other things. Hoping to get
more done next release, but meantime here's an update to the new kernel.
Tested against qemu 1.6.1, the usual suspects seem to work fine so far.)

September 17, 2013

Aboriginal Linux 1.2.5
is out, based on hg 1627 and using Linux for Workgroups
(3.11) and toybox 0.4.6.

Same versions of the other packages as last time, even busybox and uClibc.
The musl migration is coming but didn't quite make it this release.
The busybox to toybox migration continues, with toybox now supplying grep and
umount. This leaves busybox providing:

Took a while to work around QEMU's issues with arm, and sparc, and
sh4, but it's got workarounds for all of them now. That means x86, x86_64,
mips, mipsel, sparc, sh4, powerpc, and arm all work. (And sparc can actually
do native builds now, under qemu 1.5.)

The mips64 target is bit-rotted (I think qemu changed out from under it
somewhere between 1.0 and 1.2), and m68k support in qemu still isn't finished.
Todo items.

Meanwhile, toybox replaced 4 more commands in the build (mv, stat, ifconfig,
and split). Upcoming work: I've done part of the architecture to put
simple-root-filesystem in initramfs (or initmpfs)
always and splice the native-compiler in at runtime from
/dev/hda with symlinks, that might be ready next release. And I'm chipping
away at the switch to musl-libc.org (basically a complete rewrite of
sources/toys/ccwrap.c to _not_ assume uClibc).

April 2, 2013

A long over due release:
1.2.3 is based on
hg commit 1589, using busybox 1.21.0,
toybox 0.4.4, and linux 3.8.

The delay is because qemu 1.3.0 and 1.4.0 both have a
subtle bug
causing intermittent instability in the native build on all targets,
which I blamed on everything from toybox to the kernel until working
out that if I went back _two_ releases to qemu 1.2.0, Linux From Scratch built
reliably on the various targets. (A bug that
goes away when you try again is always a pain to track down, you're
never quite sure whether you've fixed it or just didn't see it this
time...)

The about page got a major facelift, if you haven't
noticed you might want to give a look.

December 16, 2012

Another kernel, another release:
1.2.2 is out based on
hg commit 1576. It's got Linux 3.7
and toybox 0.4.2.

The native-built binaries for dropbear and strace and such are back.
(Last time around native building couldn't upload binaries to the host
because the launcher script was calling busybox netcat to host ftpd,
but toybox is providing netcat now. I punted because we were close to
the next kernel release and I wanted to catch up.)

Other than the package upgrades, the rest is pretty minor.
The screenshots directory is back. Undid some
minor bit-rot in the i686-virtio target, and a tweak to powerpc so musl can
build natively on it. (Haven't switched over any targets to build musl on
the host yet, still chipping away at a ccwrap rewrite.) The
chroot-splice.sh script is now doing its bind mounts read only.

November 15, 2012

The 1.2.1 release is out,
based on hg commit 1560, with linux 3.6
(plus some ext4 bugfixes), busybox 1.20.2, and a uClibc config tweak so mdadm
can auto-scan arrays. Zlib is now at 1.2.7 (used by squashfs in
host-tools). Toybox and binutils also got updates, as described below.

Toybox was upgraded to 0.4.1 and the build is now using it by default.
The goal is to eventually replace busybox with toybox, and each release
from here on is likely to switch over a few more commands. The current
list is:

Binutils has been
upgraded to the last GPLv2 version in the git repo
(commit 397a64b3, between 2.17 and 2.18), which has the -Bsymbolic-functions
command required to build
the musl C library.
(This version of binutils also claims to have ARMv7 support, but we're not
using that yet.)

The goal for next release is to replace uClibc with musl for the
architectures musl currently supports, and to help the musl developers add
support for the remaining architectures to eventually replace uClibc as
a cross-compiled package. (New build control images to natively build
uClibc and glibc chroot environments on an arbitrary target are on the
todo list. Supporting a libc does not require cross compiling it.)

On the infrastructure side of things, USE_ALT went away. The new way to
build alternate versions of packages is to just provide a directory of
extracted source in the "packages" directory, in which case the build will
use it verbatim (applying patches is your problem).

If you want to go back to building the tarball versions, either set
IGNORE_REPOS=busybox (or all), or "rm -rf packages/busybox".

August 22, 2012

After far too long the
1.2.0 release is out, based
on hg commit 1535.
Package upgrades are Linux 3.5, uClibc-0.9.33.2, busybox 1.20.0,
toybox is back in at version 0.4.0, and most targets are finally
using NPTL instead of pthreads.

Several "how did that ever work" bugs in the build scripts got fixed
(commits 1513 through 1518, for example). Several other "only occurs on
a specific distro" bugs also got fixed (a Gentoo one in 1518, a Fedora
one in 1532). The longstanding uClibc++ build break where it complained
".config is a directory" was squashed by 1533.

The new development was mostly prep work for toybox integration and fixing
bit-rot in record-commands.sh (used to test toybox). There was also a lot of
dealing with regressions introduced by package upgrades. (Upgrading uClibc
broke stuff. Upgrading busybox broke stuff. Upgrading the kernel
broke stuff all three times.)

Busybox is still doing most of the heavy lifting both during the host
build and on the target, but the baseconfig-busybox file is now sorted with
toybox-supplied functionality at the end, allowing toybox to replace
busybox code incrementally as commands become ready.

In future versions I'm looking at migrating this project from busybox/uClibc
to toybox/musl, producing something compatible with Android's "no GPL in
userspace". I'm also looking at replacement toolchains, since it may take
me a while to clear time to work on qcc.
If you're curious about all that, ask on the mailing list.

The x86 targets now require QEMU 1.0 (because the "qemu" binary got renamed
"qemu-system-i386"), but the mips targets require the previous
release due to a regression in the network card emulation (fixed in the source
repo, but no 1.0.1 is out yet).

On the documentation front, the 2008 "what are we doing and why"
presentation is now available in
HTML format, probably worth reading if you haven't yet.
The FAQ also grew some newentries (showing how to use more/test.sh
to rebuild uClibc without redoing a while toolchain, and why powerpc isn't
as backward compatable as you'd expect).

Busybox is back to using a specified configuration instead of just switching
on everything. (Busybox's new ubifs commands won't build on Ubuntu 11.10,
and it's hard to see the command count grow past 300 and keep going without
humming the "Katamari Damacy" theme.) If we missed a command let
us know, and you can of course still build busybox defconfig natively.
(Eventually, we'll start using toybox again,
which has been relaunched as a BSD-licensed project.)

The build.sh wrapper can now rebuild from any stage, just specify the
starting point on the command line after the architecture, for example
"./build.sh i686 simple-root-filesystem" won't rebuild the cross or native
compilers, or the kernel, but will rebuild the root-filesystem, root-image,
and system-image stages.

The more/test.sh script now understands "host" as an architecture, for
testing host-tools builds.

Rename the USE_UNSTABLE variable to USE_ALT since it already uses alt-
prefixes on the packages and patches. The repository's
build/patches directory no longer contains "alt-*.patch" symlinks, instead
a new more/alt-patches.sh script can set them up locally for a given package.

Add record-commands wrapper to target image, "record-commands /mnt/init" runs
/mnt/init and logs commands to "/tmp/record-commands-log.txt". This way
you can see what commands target builds are running.

The CPUS calculation, for automatically parallelizing the build on SMP
systems, now detects hyper-threading and won't stress hyper-threaded systems
quite so hard as full SMP. The target's sbin/init.sh also has basic CPUS
detection code now too, since KVM or a chroot can have more than one. (This only
matters if CPUS isn't set by something else, such as the dev-environment.sh
wrapper script's distcc setup.)

Several bugfixes, due to insufficient regression testing of previous releases:
SYSIMAGE_TYPE=ext2 SYSIMAGE_HDA_MEGS was broken, more/smoketest-all.sh was
looking for static-tools output in the old place, and more/record-command.sh
only worked with NO_SANITIZE_ENVIRONMENT=1. The target root filesystem's
init script wasn't detecting a failure to mount devtmpfs (such as when used
to setup a chroot on a kernel that didn't support it), and falling back to
mdev -s. And ldd was spewing debug messages. All fixed now.

October 2, 2011

The big news is that Linux From Scratch 6.7 now builds to completion
under 11 targets, via the lfs-bootstrapbuild control
image. The remaining platforms all show target-specific issues
(often just insufficient board support in QEMU) rather than generic
problems with the build environment. The big bugfix allowing this to happen
was the m4 build hang (which was a bash bug).

The only upgraded package was BusyBox 1.19.2, but there were several
performance tweaks and bugfixes, such as:

The more/chroot-splice.sh script sets up a chroot using the
build/root-filesystem-$TARGET directory and a build control image,
for quickly testing build control images natively (generally using the
i686 or x86-64 root filesystems).

The more/buildall-native.sh timeout value is now configurable via the
TIMEOUT=seconds environment variable (defaulting to 60).

Tuned the
more/buildall.sh script in FORK=1 mode to take available memory into account
determinging CPUS (so it doesn't overload the build machine so easily).

When
dev-environment.sh enables distcc it sets CPUS=3 because several current QEMU
boards support a maximum of 256 megs physical memory.

The ldd and ldconfig binaries in the cross compilers were target binaries
instead of host binaries: that's fixed now.

The virtio targets should now work with dev-environment.sh

The test script more/test.sh should no longer delete the output directory
(build/$STAGE_NAME-$TARGET) at the start of each test, allowing test rebuilds
of things like uClibc which have target prerequisites.

The common kernel config now uses the ext4 driver for both ext3 and ext2
filesystems.

August 22, 2011

The
1.0.3 release is out,
based on hg commit
1430.
This release includes the long-awaited Linux 3.0 and uClibc 0.9.32
releases, along with BusyBox 1.19.0 (and Dropbear 0.53.1 in the static
tools).

This release is still using pthreads, but NTPL support is a goal for next
release. (It's just a uClibc config change.)

The Aboriginal Linux project is feature complete ever since the 1.0
release, and other than adding support for more targets the only major
todo items left are a big documentation update, more automation via cron job,
and enabling NPTL support. (This release is still using pthreads, but NPTL
support is a goal for next release. It's just a uClibc config change at
this point, 0.9.32 has support but needs more tweaking and testing.)

Since control images are not feature complete (and there's really no obvious
limit to what you can do with them), they deserve to be their own project.

Other changes:

The default kernel configuration replaces includes v9fs (as used in virtfs or
the diod server) instead of the older NFS. In addition Alessio Igor Bogani
contributed i686-virtio and powerpc-virtio targets, using the new virtio
infrastructure in KVM to provide efficient emulation-friendly virtual
devices.

Several random bugfixes and cleanups (BINARY_PACKAGE_TARBALLS should
work again, the root filesystems now use devtmpfs instead of mdev).

Major cleanup on sources/targets, now most entries are a single file.
The hold hw- prefix stuff went away, now the ones that are files are built
by more/buildall.sh and the ones that are directories are the derived stuff
that only gets built if you specify it explicitly. (read_arch_dir is
now called load_target, and the new base_architecture function reads in
a base architecture that this one derives from.)

The new more/smoketest-report.sh produces CSV output with more info.
(I need to follow this up with a nightly cron job and a pretty
web page generator.)

June 14, 2011

This release includes several bugfixes, package version upgrades for Linux
(2.6.39), BusyBox (1.18.4), and Squashfs (4.2), and some minor user interface
tweaks. (Still using uClibc 0.9.31, NTPL is bumped to next release.)

Targets

This moves forward three(!) kernel releases (sorry), and although several
targets bit-rotted along the way (notably armv6l, sh4, and mips) they're all
fixed now, including some that didn't work last release.

QEMU still doesn't emulate the big endian arm or
m68k targets, powerpc-440fp needs to be switched to QEMU's -M bamboo board
emulation now that it can actually emulate a 440, and mips64 and sparc boot
to a shell prompt but contain bugs in either uClibc or the kernel that make
them unreliable.

All the other targets can at least natively compile and run
"hello world", and the arm, x86, mips, and powerpc targets built
static versions of dropbear and strace (available in the
extras directory, or by using
native-build.sh with the static-tools.hdc control image).

New Control Knobs

Two new build stages root-image.sh and linux-kernel.sh
got split out from system-image.sh. The first
makes a filesystem image from a root filesystem directory,
the second builds a kernel. This leaves system-image.sh to
create the qemu launch scripts and bundle it all up into a tarball.

The advantage of the split is you can repackage the root filesystem without rebuilding
your kernel. (Unless you chose SYSIMAGE_TYPE=initramfs, where the output
of root-image.sh is a prerequisite to linux-kernel.sh.) The downside is
you have to remember to call the other scripts before calling system-image.sh
in order for your changes to wind up in the system image. (If you delete
the tarball you modified out of the build directory, ./build.sh should still work out
what it needs to rebuild.)

Environment variables: SYSIMAGE_HDA_MEGS now defaults to 2 gigabytes
(only currently matters for SYSIMAGE_TYPE=ext2). The more/buildall.sh script
now pays attention to NO_CLEAN and BUILD_NATIVE_ONLY. As always, the
environment variables are documented in the config file at
the top of the source tree, and leaving them blank should provide sane
default behavior.

Two new scripts in the system image's /sbin directory are:

setup-chroot /home/mydir [COMMAND...]

This script copies the system-image's (generally read-only) root filesystem
into a writeable subdirectory, copies the other mounts ("mount --bind /proc
/home/mydir/proc" and so on), and chroots into the resulting directory to
run the rest of the command line. (If no commands are specified, chroot
runs /bin/sh for an interactive shell prompt.)

When the chroot exits, the script cleans up all the mount points but
leaves the directory contents.

This lets you use a read-only system image to build and install software
into a chroot.

If the target directory already exists, the script will just redo the
--bind mounts, and won't copy files.

zap-chroot - The cleanup script to remove all mounts under a
specified subdirectory.

In the system images, root's home directory is now /home/root so it's
in known writeable space you can stick a .ssh directory in for dropbear.

February 13, 2011

The repository is back up, and tweaked so it generates correct URLs without
superfluous cgi names in them.

I see from the release RSS feed (go to repository viewer, then click "tags"
and then "rss") that we're due for the next release in the first half of March.

I'd better get on that. (I'm aiming for a version of uClibc with NPTL
working on at least x86/64, arm, mips, and powerpc. There's some work to do
between now and then...)

February 8, 2011

The website was down again while we switched web hosts, but we're back up
on dreamhost now, which has buckets more bandwidth and lets us have a
mailing
list again.

I still need to figure out how to enable mercurial on the new host, so
the repository is down at the moment, but I should get that fixed up
soon.

January 2, 2011

Automated native build control images

The big news this time is the native build
control images, especially
the one to build Linux From Scratch 6.7.

These control images drive the automated build infrastructure (first
introduced back around release 0.9.8 and substantially improved since then).
They allow you to fire up a system image under QEMU and run a completely
automated native build.

To use a control image, cd into a system-image
directory and run "./native-build.sh path/to/control-image.hdc". The
result should wind up in the "uploads" subdirectory. To build control images
from source, run "more/build-control-images.sh" and look in
"build/control-images" afterwards. The build scripts live in
"sources/control-images".

The lfs-bootstrap.hdc
image builds most of Linux From Scratch 6.7.
It doesn't replace the toolchain or C library, and stubs out
internationalization support, but builds everything else (about 50 packages
total). This build runs to completion reliably on i686, x86_64, and
armv5l. Making it do that smoked out a lot of subtle bugs and configuration
issues in the build environment, resulting in much improved system images.
This LFS build sometimes finishes on mips and powerpc, and
sometimes randomly dies for various asynchronous reasons that seem like
something wrong in uClibc's pthreads locking code, although that's just
a guess. (Since the uClibc developers (finally!) shipped a 0.9.32-rc1 with NPTL
support on all targets, I expect the fix will involve upgrading. Part of the
delay in this release was me playing around with that, but I'm releasing a
checkpoint of the existing code before switching over to the new threading
implementation.)

What else is new?

I fixed the static "dropbear" binary to call "ssh" out of $PATH instead of
expecting dbclient at a fixed location, fixed the guest account's
/etc/passwd and /etc/group entries so dropbear can use it, and added a /root
directory to the system images so you can add a .ssh directory to it to make
dropbear happy (possibly by mounting a tmpfs on it first).

Build infrastructure tweaks: sources/download_functions.sh is now its own
file, and various fixes (only apply patches that end in ".patch") . The root
filesystem's sbin/init.sh now mounts /dev/pts. Bugfix for
SIMPLE_ROOT_OVERLAY, and multiple bugfixes in ccwrap.

Lots of tweaks to the native-build stuff. Redo control image generation,
add bootstrap-skeleton with common code for bootstrapping distros (it has
its own README), and rebase
the half-finished gentoo-bootstrap on top of that. Upgrade
"bisectinate" debugging script to take its test from the command line.

Several tweaks to uClibc to build LFS packages, and tweaks to the
base kernel config to support NFSv3 and devtmpfs.

The new more/chroot-splice.sh script combines a root filesystem directory
and a control image directory into a combined version you can chroot into
and run a build from. (It looks for each in the location the build puts
them, build/root-filesystem-$ARCH and build/control-images/$IMAGENAME
respectively.) The new more/zapchroot.sh script unmounts everything under a
directory, which is useful when playing with a root-filesystem via chroot.

Target support

I've fallen a bit behind on regression testing the targets, and hope to
catch up next release. (Upgrading the kernel and QEMU breaks things that used
to work.)

Last I checked, qemu still didn't support armv4eb or m68k, so I can't
test those, but they build. (The kernel .config is almost certainly wrong
for any target you'll actually be able to run them on, but the toolchain
and root filesystem are probably ok.)

Updating the kernel broke armv6l: attempting to boot it under qemu exits
with an "unimplemented cp15 register write" error. I'm not sure whether this is
a kernel bug, a qemu bug the upgrade triggered, or some kind of
misconfiguration. The proper fix is probably to look at the other arm boards
QEMU can emulate for newer processors, and switch to one of those.

QEMU had a MIPS bug that prevented the system images from working for a
version or two (such as the one in Ubuntu 10.04). It's fixed in current
qemu-git, and isn't actually a problem with the mips, mipsel, or mips64 system
images, they work fine under older or newer versions of qemu.

Sparc broke again (failure to launch init), and Oracle's patent trolling
makes it harder than ever to care. It's on the todo list...

The only system image that doesn't current compile is sh4. (As noted in the
0.9.11 release notes, the sh4 maintainer broke it on purpose, which also makes
it hard for me to care much about that target.)

The problem with sh4 is once again the kernel, this time a build break:

And even if I bothered to fix that, another change in the sh4 code changed
the serial devices so QEMU's serial console no longer produces output. (Note
that the sh4 test image on qemu.org uses a 2.6.29 kernel, which may be the last
time the thing actually worked for anybody who isn't in a financial relationship
with Renesas.)

Last I checked, QEMU still didn't support armv4eb or m68k, so I can't
test those. I have patches to make m68k work on aranym and armv6eb work on
actual big endian arm hardware, but haven't merged them yet.

November 3, 2010

My apologies for the lack of notice, the impactlinux.com domain unexpectedly
went away saturday. The landley.net mirror is still up, and is
becoming the main site again for the moment despite its' limited bandwidth.
I'll see if I can recover the mailing list subscriptions, but I wouldn't
count on it.

Still working on the next release, currently scheduled for around the start
of December. Until we get a new mailing list, the
source control log shows what's changing, and
you can always email me directly, rob at landley dot net.

September 5, 2010

There's an old saying, "eventually you need to shoot the
engineers and go into production". Aboriginal Linux can now bootstrap a
native development environment on an arbitrary target, and build packages
under the result. There's always more to do, but it does what it set
out to do. And thus:

Yes, it's been over five months since the last release. I didn't want to
ship yet another intermediate release before doing 1.0, but I was trying
to run out of todo items before shipping. (Yeah, I should know better.
Sorry for the delay.)

The following are release notes, not proper documentation. Proper
documentation (including an introduction to the whole thing) is the next
big TODO item.

Targets

The arm, mips, powerpc, and x86 targets all work. Sparc works statically
linked, but dynamic linking is still broken. Khem Raj fixed mips64 dynamic
linking, but the native compiler still doesn't work. The m68k target builds
again, although QEMU still doesn't emulate it. The sh4 target and big endian
arm are are still broken.

The snapshots page needs updating. The documentation needs updating.

System image updates

The native-build.sh script moved into each system image. This means you
can now download a system image tarball and a control image, then run a
native build without any dependencies on the Aboriginal Linux build scripts.
This script is a wrapper around dev-environment.sh which takes one argument,
the name of the control image file.

The build only puts dev-environment.sh and run-from-build.sh into a
system image when the system image includes native development tools.

Control Images

This release adds prebuilt binary build control images, generated
by more/build-control-images.sh and shipped in the
control-images directory.
These are squashfs filesystems containing source code and an init script to
build it and upload it to the host (using the $FTP_SERVER and $FTP_PORT
variables, set by native-build.sh).

If these
images are loaded into /dev/hdc of the emulated system, the system image's
init script will mount that on /mnt and run /mnt/init instead of a shell
prompt, so the build is completely automated.

Control images are provided to build hello world, build static dropbear
and strace binaries, run the busybox test suite, and an in-progress script
to bootstrapping Gentoo Stage 1 does about the first 2/3 of the job.

Root filesystem improvements

The root filesystem broke into two stages: simple-root-filesystem just
contains a simple "defconfig busybox plus uClibc" system, enough to boot
to a shell prompt. The native-compiler stage is like cross-compiler only
unprefixed, built to run on the target, and including make, bash, and
distcc. The root-filesystem stage combines the two into a single root
filesystem with native development tools.

If the root filesystem is mounted read only, the init script will now
mount a tmpfs on /home if there's no /dev/hdb to provide writeable scratch
space. Chroot environments should now use Google's 8.8.8.8 nameserver
by default. The distcc binaries moved out of /bin to the /distcc directory.

The "oneit" binary became a standalone program, and uses the / directory
as PID 1's working directory, allowing you to umount /home if necessary.
Toybox's "patch" command went upstream into busybox, and busybox's defconfig
once again supports netcat server mode, so toybox was removed. We're now
using a defconfig busybox, so configuring it properly is their problem.

Build scripts

The kernel configuration got collated to sources/baseconfig-linux (just
like baseconfig-uClibc), and the various packages were migrated to use
that. The kernel config is now the baseconfig plus the settings
entry $LINUX_CONFIG. You can still supply a miniconfig-linux file to
override this, and some architectures still do that. The ones that have
been migrated to use a shared baseconfig are a lot more regular now,
and the differences between architectures broken out and isolated.

The build stages were broken up to be more granular. The root-filesystem.sh
stage now merely collates the output of two previous stages:
simple-root-filesystem.sh (which produces just enough of a root filesystem
to boot to a shell prompt, based on uClibc and busybox plus a few init scripts
and config files), and native-compiler.sh (a portable native toolchain).

The system-image.sh stage now generates the dev-environment.sh script.
It only puts dev-environment.sh and native-build.sh scripts in the tarball
when NO_NATIVE_COMPILER is blank (and thus the build includes native
development tools), or if root-filesystem isn't available in which case
it'll automatically use simple-root-filesystem.

Several more package builds were moved to sources/sections.

The new "more" subdirectory collects together all the user-runnable
scripts that aren't build stages. (Many of these were previously in
sources/more.) New scripts in this directory include:

Scripts to run newly built system images out of the build directory
("run-emulator-from-build.sh $ARCH", "dev-environment-from-build.sh $ARCH",
and "native-build-from-build.sh")

timeout.sh $SECONDS command... - Wrapper to run a child process and
kill it if it doesn't produce a line of output for $SECONDS seconds.

build-control-images.sh - build all control images from sources/native-builds.

for-each-target.sh - iterate through every defined target architecture
and run a command.

Website

A new cron job is populating the
snapshots directory. Currently only
the stable versions, but the unstable versions should wander back in
eventually. (When the bisectinate and migrate-kernel scripts catch up to
what they need to do to properly test unstable packages.)

More documentation: The "about" page got redone, and the FAQ was fluffed
out with a new debugging section. Michael S. Zick wrote up mips docs for the
architecture page, and the build scripts grew even more comments.

General Infrastructure

The control image generation was upgraded so each control image
source can have its own patch directory and file directory. (This needs
more work.)

Environment variable sanitization: the sanitize_environment function
unsets any environment variable it doesn't recognize, either from a whitelist
or from config. Clearing all unrecognized environment variables insulates
the build from some of the strange environments out there. To disable this
behavior, export NO_SANITIZE_ENVIRONMENT=1.

New config variables:

STATIC_CC_HOST became CROSS_HOST_ARCH, set that and second stage cross
compiler gets built.

The new ALLOW_PATCH_FAILURE=1 variable continues despite a patch failing to
apply.

NO_HOST_TOOLS=1 tells build.sh to skip the host tools step, which is seldom
a good idea but there you have it. The new HOST_EXTRA is an equally horrible
idea, but if you feel the need it's a list of extra commands to pass through
from the host $PATH.

NO_CPLUSPLUS=1 tells it not to add C++ support in the toolchains it
builds.

NO_CLEANUP isn't a new variable, but you can now set this for entire
builds and it should work.

ccwrap improvements: Handle unexported $PATH better. It doesn't need to
search $PATH when argc[0] is a path (either absolute or relative to the current
directory). Also use Scrt1.o when building a shared library. (Oddly this
never caused a problem before, but do the right thing anyway.)

Upgrades to BUILD_STATIC: it now takes a comma separated list of packages,
with the special value "all" to build everything static, and "none" to build
nothing static. The default is to build static versions of busybox and the
toolchain packages (gcc, binutils, make, and ccwrap) both due to the increased
performance this provides under qemu (which has to repeatedly re-translate
executable code pages that modify themselves, such as when dynamic linking
patches per-process jump addresses), and also due to the increased portability
this provides the native compiler tarball.

(The host-tools stage always builds dynamic packages, because the glibc
maintainer has a personal dislike of static linking, meaning glibc's static
linking support (in addition to creating 400k "hello world" binaries) is both
broken[LINK] and deprecated[LINK]. Some distros have even taken to breaking
out the *.a format libraries used for static linking into a separate package,
and not installing them by default. The host-tools stage now tests for this,
and refuses to continue in a dynamic-only build environment unless you set
BUILD_STATIC=none. You can rebuild under the resulting target system to
natively compile static binaries, which uClibc supports quite well, but you
can't generate them from the host.)

The old FROM_ARCH is now HOST_ARCH, and the infrastructure using it
(cross-compiler.sh and native-compiler.sh) has been cleaned up. The
HOST_UTILS variable was also removed, that's now built for the full
cross and native compilers, but not the simple cross compiler.

The more/smoketest.sh file no longer creates hdb.img.

The host-tools stage has been upgraded to work around the utter stupidity of
Ubuntu 10.04, which turned "gcc" into a perl script. (Yes really. No, I don't
know what they were smoking.)

Build zlib in host-tools (for squashfs) and in static-tools for dropbear's
-C. This is necessary for reproducing the build under a system-image, because
the default system image type (squashfs) requires zlib to build.

Update the command prompt in system images to show the amount of
parallelism available via distcc. (When it says 1 you haven't got
distcc, and when it says a number greater than that you have.)
The prompt now looks like:

(powerpc:4) /home #

The make, bash, and distcc builds are now part of the native-compiler stage
(and their build scripts moved to sources/sections). This means that a native
toolchain now includes make, because it's hard to build make without make.
This stage also records the uClibc and linux config files for posterity.

The build was weaned off of toybox. (Broke out oneit, sent patch upstream
to busybox, and got netcat fixed in defconfig so native-build.sh could use
it.)

Several small bugfixes and cosmetic cleanups. For example, we now delete
empty build/temp-* directories when we're done with them, and several
config options such as NO_NATIVE_COMPILER and NO_CLEANUP should be more
reliable now. Don't build hello-dynamic on a BUILD_STATIC=all system.
Clean up debris files the kernel's headers_install leaves lying around.
Minor upgrades to FORK=1 support, and QUIET is now part of FORK instead
of a separate control. Generating a package manifest should no
longer produce error messages. Empty temp directories get deleted on the
way out. Better support for building older tools as alt-packages (tested
binutils 2.14 and gcc 3.4). Comment out default UNSTABLE= URLs in download.sh,
supply your own if you want that. Binutils build now honors existing $CFLAGS
(if any). The dienow error message goes to stderr. More reliably track
package cache state when patches fail to apply. Simplify system-image
build (don't try to package root filesystem and build kernel in parallel
anymore, squashfs and make both have internal SMP support). BUILD_VERBOSE
is now more verbose. Reliability improvement for record-commands.sh.
New shell functions blank_workdir and extract_package. Consistently
use name-name in scripts and name_name in functions.

It's been a long 5 months...

May 2, 2010

The name of the project is changing from Firmware Linux to Aboriginal
Linux. The new URL is "http://landley.net/aboriginal".

Some reasons to move away from the old name are
listed here,
and some reasons to pick the new name are covered in
this thread.

Some links may be broken while we switch over. Let us know what we've
missed on the mailing list...

Improved native builds

These dropbear and strace binaries are built using the automated three-disk
native build capability introduced back in November,
which has been improved and genericized this release. Now one script creates
a build image (see examples in sources/native-builds) and another script
launches a system image with that build image attached as the third
hard drive (/dev/hdc), telling the system image to automatically build
those source packages and upload the result to the host via ftp.

So to reproduce the dropbear and strace binaries for a supported ARCH,
you could do:

When the build exits, the results should be in the host's "output"
directory.

Targets

The arm, mips, x86, x86-64 targets that worked last time still work.
The armv4eb and m68k targets still don't work.

The powerpc and sparc targets have been upgraded. Powerpc now works with
the 3-disk automated native build (although you'll need qemu 0.12.4 to take
advantage of this). The sparc dynamic linker still doesn't work in uClibc,
but the statically linked system is now working well enough to build packages
natively.

New targets this release are i486 (thanks David Seikel) and mips64 (which
mostly works, but the native toolchain isn't quite properly configured to
build stuff with yet).

Finally, the sh4 target was broken by a cosmetic cleanup of the startup
assembly code in Linux 2.6.33 that rendered any binutils version older
than 4 months unusable to build sh4. While it's possible to patch around
this, it hasn't yet been worth effort. The Linux-sh4 maintainer is aware
of the issue, but explicitly
doesn't
care about users other than his employer, and so far nobody
else seems to care enough about supporting the discontinued Sega Dreamcast
game console to argue with him. (Might revisit this next release, but m68k
support is a higher priority.)

Infrastructure

The system-image's "run-emulator.sh" is now a much smaller and simpler
script, consisting primarily of the qemu command line. (And if you export
DEBUG=1 it prints the emulator command line.) The distcc and hdb
setup code moved to dev-environment.sh.

The bisectinate.sh script (to automatically git bisect a build issue)
has been updated, more to come next release.

The download.sh script has been updated to understand FORK=1 and download
packages in parallel. (And to extract them in parallel when
EXTRACT_ALL=1.)

The new export_if_blank shell function allows several more environment
variables to be overridden, including TOP, SOURCES, SRCDIR, PATCHDIR,
BUILD, SRCTREE, HOSTTOOLS, WRAPDIR, CC, and STAGE_NAME.

I'd also like to thank the debugging efforts of Wolfgang Denk, who hit
as many bugs trying to use this project as I normally do with other people's.
He eventually got discouraged and left, but thanks to his efforts:

We now unset several environment variables (such as ARCH and
CROSS_COMPILE) that break things if the host exports random data in them
before the build. (Yes, this fights with export_if_blank above, and
the way configuration works in general. Some more design works is necessary
here to let you configure the project with environment variables, but
not be confused by existing environment debris.)

BUILD_STATIC=none can how handle the case where the host toolchain
can't build any static binaries at all.

The CROSS_SMOKE_TEST implementation (which had bit rotted) was fixed and
moved to its own script (sources/more/cross-smoke-test.sh), which is called by
build.sh as necessary.

Note that actually trying to use it is a question
of getting qemu application emulation to work for your target. For
arm you need to tweak /proc/sys/vm/mmap_min_addr as root. For ppc you
need this patch. For mips64 somebody still needs to
implement it (no matter what gentoo thinks).

(There's a reason the smoke test was disabled long enough to bit-rot.
It broke for qemu reasons more often than it found bugs in the toolchain.)

February 2, 2010

Yeah, I know 1.0 is overdue for a release, here's a resync point
with the 2.6.32 kernel and uClibc 0.9.30.2.

There are some regressions this time around, which I plan to fix up
in the next release. The 2.6.32 kernel broke strace,
and upgrading to the new version didn't fix it. (I know how, I just haven't
yet.) The m68k target isn't currently building due to the uClibc upgrade
(yeah, "bugfix only", I know), and the armv6l target seems to be broken two
different ways. I screwed up the screenshots page (which is halfway converted
to Vladimir's new fancy version with style sheets, but "halfway" doesn't
actually display right). The cron job building nightly snapshots got taken down
when the server's hard drive filled up, and now it's in the process of being
upgraded with a couple new scripts:

The new sources/more/migrate_kernel.sh script does a "make oldconfig"
between the stable and "USE_UNSTABLE=linux" kernels, and creates a new
alt-miniconfig-linux for the specified target with any new symbols that
showed up. (This means the snapshots should break less, but adding the
symbols to the native configs will still require manual filtering.)

Also, the new sources/more/bisectinate.sh script automatically does
a git bisect between a known good and known bad (generally HEAD) version
of one of the unstable packages (linux, uClibc, or busybox). It can test
that the build completes, or that the system image boots and successfully
builds the static native dropbear binary. (In theory this will allow the
nightly snapshots to automatically isolate and report the first commit that
broke them.)

In other general infrastructure news: some extensive upgrades to the
compiler wrapper (ccwrap), such as no longer needing GIMME_AN_S at compile
time and instead autodetecting the presence of libgcc_s.so vs libgcc.a
and behaving appropriately. A largish cleanup/refactoring of the compiler
build (described in
commit 944)
added a new "native-compiler.sh" script, which handles building the
improved "cross-compiler" tarball that includes thread support and
uClibc++ and is statically linked against uClibc on the host.
The simple cross compiler is now "simple-cross-compiler.sh" (which is
good enough to build a system image, but isn't very useful in other
contexts).

The host-tools.sh stage now only builds
e2fsprogs or squashfs if the build is actually going to use them (based
on the SYSIMAGE_TYPE config symbol). The sources/sections scripts now
come in two flavors: an *.build version with setup/cleanup called
automatically and *.sh files which do their own setup/cleanup within
the script. Each system image's run-emulator.sh will once again add
/sbin:/usr/sbin to the $PATH if it can't find mke2fs.

The --extract command line option went away from download.sh because
everything else works via environment variables. Use EXTRACT_ALL=1 instead.

Fixed a few regressions that snuck into previous versions: the
config-{linux,uClibc,busybox} should once again be in the root filesystem's
usr/src directory, buildall.sh should now try to build the static native
dropbear/strace binaries for all targets (not just the last one) and it
should properly be running smoketest.sh again.

Added shm support to uClibc. Bugfix to the c++
compiler (it should work again). The system images now have a "guest"
user/group so packages can more easily be built as a non-root user.
The shell prompt now includes the $HOST name.

Build scripts

A new trivial-but-convenient "./clean.sh" script lets you delete just the
target builds out of "build". (If you delete that whole directory it has to
re-extract all the package tarballs and rebuilt the host tools, which is
often unnecessary work.)

Command recording has been completely redone. No more RECORD_COMMANDS
variable, instead run sources/more/record-commands.sh to wrap the current
$PATH in build/wrappy (either after or instead of running host-tools.sh),
then run your build stages, and finally either run
sources/more/report-recorded-commands.sh to get a summary or
look at the raw command line lists in build/logs.

The Great Refactoring continues in sources/sections: broke up
binutils-gcc.sh into individual binutils, gcc, and ccwrap
stages, split off linux-headers and uClibc++ from uClibc.sh,
and spun off toybox.sh from busybox.sh.

Also broke kernel_cmdline() out of qemu_defaults() to make it easier to add
non-qemu sources/targets configurations without reinventing the wheel,
and fixed up dependencies for hw-target builds (which should work a lot
more reliably now).

Bumped up the number of $CPUS used by the build to 1.5x the actual number
(as long as your host has at least 512 megs of ram per actual CPU). That
should help keep larger servers busy.

Some self-hosting fixes from Natanael Copa (making sure host-tools.sh
builds on a uClibc host): added --disable-nls to the e2fsprogs build
(what exactly is ./configure running all these tests for again?), and a
squashfs patch (substituting a function uClibc hasn't got).

Simplified system-image.sh to use killtree() instead of mysetsid.
Simplified the cross-compiler/cross-static logic to set the $PATH to
use the right one, rather than moving them around during the build.
Updated smoketest-all.sh to show "NONE" for system images that didn't build.
Added more support for SKIP_STRIP and CFLAGS=-g to the rest of the
packages. Cleaned out the old USE_COLOR stuff which was superseded by
set_titlebar.

November 7, 2009

This release upgrades linux to 2.6.31.4 and busybox to 1.15.2.
(The uClibc version is still 0.9.30.1, they haven't had a new release
yet.)

The old #firmware irc channel on freenode.net has been merged into #edev,
so go there now to ask questions about this project. (A _lot_ of
knowledgeable, helpful people hang there who can answer your embedded
development questions.)

Bug tracker and roadmap

It's also a place to file bugs, although sending them to the mailing list
is still the fastest way to get a fix.

New screenshots page

This page lists every target, with
a "screenshot" of the target's boot messages under qemu, and associated
links to all the various prebuilt binaries for that target.

Fun with static linking

The cross-compiler-$ARCH tarballs are all compiled for i686 and
statically linked against uClibc for maximum portability.

The busybox binary in the root filesystem images is now built statically by
default, providing a 20% speed improvement in ./configure times under qemu
(due to the reduced page translation overhead). You can switch this back
to dynamic with "BUILD_STATIC=none".

Static dropbear and strace binaries for each target are also available,
built by the new sources/more/native-static-build.sh script.

System Image interface changes

The run-from-build.sh script used to set up a development environment
out of the build directory, but didn't provide an easy way to do so based
on downloaded tarballs. (You had to work out and provide extensive
command line arguments to the run-emulator.sh script.) This has been
fixed.

The run-emulator.sh script in each system interface tarball is now much
simpler. It now looks for distccd and $ARCH-cc in its $PATH, and sets up
distcc acceleration automatically if it finds them. (It announces whether
or not it has managed to do so, and init.sh announces whether or not it
thinks it has distcc acceleration enabled just before launching a command
prompt.) So setting up the distcc accelerator should be less of a pain.

Also, run-emulator.sh no longer takes any command line arguments. Instead
the environment variable $HDB indicates a /dev/hdb disk image to mount on
/home, $HDBMEGS specifies the size of a (sparse ext2) HDB image to create if
it doesn't already exist, and QEMU_MEMORY indicates how many megabytes of
physical memory the emulator should allocate for the virtual system.

Each system-image also provides a simple wrapper dev-environment.sh
which calls run-emulator.sh with HDBMEGS=2048, HDB=hdb.img, and
QEMU_MEMORY=256. (These are the values run-from-build.sh used to provide,
and give a reasonable build environment with 256 megs of physical memory
and 2 gigabytes of writeable /home space.) The run-from-build.sh script
itself now merely adds the specified build/cross-compiler-$ARCH directory
to the $PATH, changes directory to the appropriate build/system-image-$ARCH,
and calls ./dev-environment.sh. It's 3 lines.

The system images can now also automatically mount a third disk image
($HDC, specifying an /dev/hdc image to mount onto the /mnt directory), and if
that filesystem contains an executable "init" program in its root directory
the sbin/init.sh script will run that instead of launching a command prompt.

The new native-static-build.sh uses this HDC capability to automatically
run some native package builds without having to drive the build through
/dev/console via a wrapper script (the way smoketest.sh does). See
sources/more/setup-native-static-build.sh for the script which generates
the hdc.sqf image for this.

Note that some targets (such as sh4 and powerpc) don't yet support /dev/hdc
due to QEMU board emulation limitations.

Build interface changes

The new "sources/more" directory contains scripts the end user can
call, but which aren't important enough to clutter up the top level
directory with. The buildall.sh and smoketest-all.sh scripts moved there,
as did the cronjob.sh used to build the nightly snapshots.

The new sources/more/native-static-build.sh natively compiles dropbear and
strace under the emulator, and copies them out to the host through the virtual
network (into the build/cron-temp directory) using busybox ftpd on the host.
This provides an example of how to use the new HDC /mnt/init functionality
to perform automatic builds under the emulator, and copy the results out
through the network. It uses the new sources/timeout.sh wrapper to detect
hung builds (defined as builds that haven't produced a line of output within
a given number of seconds).

The new BINARY_PACKAGE_TARBALLS variable (in the file "config") tells
the build to create a separate tarball of the binaries produced building
each package. (These more granular tarballs aren't shipped yet, but may
be in a future release.)

The new SKIP_STRIP variable tells the build not to strip binaries.
(Add in CFLAGS=-g for enormous binaries full of debug info.)

Target upgrades

Powerpc and sh4 should now work fine with stock qemu 0.11.0.

The arm platform now has an armv4tl target. This is a little-endian armv4
eabi, for armv4 chips offering the "thumb" extension which eabi requires.
(The plain armv4l target is still oabi. The oabi/eabi thing is different
binary ABIs, a bit like Linux vs BSD binaries or aout vs elf. The eabi
standard is the new one, but some of the older arm hardware can't support
it. This takes it down as far as it can go.)

Infrastructure

Behind the scenes, some extensive work is going on to simplify the
build scripts. (The documentation's fallen a bit behind reality here,
but should be back up to date next release.)

The uClibc miniconfigs are now automatically generated, with the
bulk of the config living in sources/baseconfig-uClibc and the few
target specific lines being appended from the UCLIBC_CONFIG variable
defined in the target's settings file.

The other big change is factoring out lots of common code, moving it into
the sources/sections directory, so things like the uClibc, busybox, and
toolchain builds now live in one place and are called from the various
other scripts that need them. (This may eventually allow alternate build
scripts for these packages, potentially swapping out glibc for uClibc, or
llvm/clang for gcc. In the meantime, it eliminates a lot of redundant code.)

This required lots of small cleanups, such as making the scripts now
consistently use the $STAGE_DIR variable for their output directory.
Some other cleanups were primarily cosmetic, such as moving the extracted
tarball cache directory from build/sources to build/packages. (Tarballs
live in packages, extracted tarballs live in build/packages.)

The root filesystem now contains simple /etc/passwd and /etc/group files,
because dropbear won't work without them (not even to ssh out).

Several small bugfixes (ccwrap no longer segfaults if $PATH isn't
exported, /etc/resolv.conf doesn't get overwritten unnecessarily by init.sh,
etc).

August 20, 2009

Version 0.9.7 is out, a little
over a month and a half late. It's a larger update than usual, so the
release notes are a bit long.

This release is based on
mercurial version 807
of the build scripts,
and includes Linux 2.6.30.4 and BusyBox 1.14.3.
No new uClibc release is out since last time.

Lots of little bugs got fixed. For example, this release should build on
Fedora 11 (and any other platform that hasn't got "which" installed by
default), and now supports distcc and ccache on the host (and will use them
automatically if they are installed).

The old "mini-native" stage has been renamed "root-filesystem"
(the same way package-mini-native became system-image last release). The
script names (and resulting tarballs) now indicate what each stage makes.
So build.sh calls host-tools.sh, cross-compiler.sh, root-filesystem.sh,
and system-image.sh, in that order.

The old build-static-toolchains.sh script is gone, with its functionality
integrated into buildall.sh (which now builds i686 hosted static toolchains
for each target by default). This is implemented by two new environment
variables triggering optional behavior: STATIC_CROSS_COMPILER_HOST
and BUILD_STATIC_NATIVE_COMPILER.

Setting STATIC_CROSS_COMPILER_HOST to an architecture name creates a
cross compiler statically linked to run on that host (via a technique
known as canadian cross compile). For example, the invocation:

BUILD_CROSS_COMPILER_HOST=i686 ./build.sh armv4l

Would create a tarball called "cross-static-armv4l.tar.bz2" which was
built to run on an i686 host, and was statically linked (against uClibc)
on that host to be distribution independent.

Setting BUILD_STATIC_NATIVE_COMPILER to any non-empty value creates a
native compiler for the target (packaged as native-compiler-$ARCH.tar.bz2),
which is statically linked against uClibc so it can be extracted and run on
just about any system of the appropriate type. (So if you can get a Linux
shell prompt and have enough storage and memory, you should be able to
compile stuff.)

The upgraded smoketest-all.sh script runs smoketest.sh on each
target built. (The smoketest.sh script attempts to compile and run "hello
world" natively within a system image under qemu, and indicates whether or
not it worked via "pass" or "fail".) Note that smoketest-all.sh now
autodetects which targets have been built (have system images in the build
directory), and smoketest.sh times out after 60 seconds so targets that hang
will return failure eventually.

Squashfs 4.0 now the default system image type

The root filesystem image type is now squashfs by default. (To change it set
SYSIMAGE_TYPE to ext2 or initramfs, either in your environment or the file
"configure".)

This involved upgrading the userspace squashfs support package
to the version 4.0 supported by the upstream kernel, and numerous small fixes
to work with a read-only root filesystem (with writeable spaced mounted on
/home and /tmp).

Target upgrades

ARM upgrades

Lots of work on the arm targets, with the addition of a new armv6l target
and conversion of the armv5l target to
ARM EABI (which used to be documented here,
but ARM screwed up its website and now you have to go
here). The armv4l target still provides an ARM v4
little endian soft float OABI target (which is "the i386 of ARM", meaning it
should run on just about any arm hardware, if a bit slowly). A new
armv4be target is the same as armv4l except big endian.

Updated sparc support, almost works now. (uClibc for sparc still sucks.)

The sparc target has also been upgraded to the point where it now gives
a shell prompt. (And immediately dies with a bus error or a hang if you try
to do anything, because uClibc for sparc still doesn't work. The "upgrade"
consisted of statically linking everything because uClibc's dynamic loader
doesn't work for sparc.)

PowerPC remains fiddly due to QEMU issues, new powerpc-440fp target

PowerPC support in QEMU is
officially unstable, so although FWL's powerpc target hasn't significantly
changed from last release, the version of QEMU in source control no longer
runs it (nor the binary powerpc system images released with FWL 0.9.6) and
QEMU's developers do not consider this a regression. While we search for a new
Linux kernel .config that matches QEMU's current expectations (and await a
release considered "stable"), note that setting HOST_BUILD_EXTRA=1 before
running host-tools.sh will build a version of qemu that does run powerpc and
put it in build/host where run-from-build.sh can find it. (Note also that
building qemu from source still takes significantly longer than the rest of the
host-tools.sh stage combined.)

A new powerpc-440fp target produces code for the powerpc 440 with hardware
floating point. (Theoretically QEMU's "bamboo" board emulation should
run this, but QEMU doesn't emulate a 440 processor yet. We compromise by
running a ppc440 filesystem under qemu's power mac emulation, to take
advantage of the ppc440 instruction set being almost a subset of full powerpc.
This gets us a shell prompt, but throws illegal instruction errors if you
try to do anything fancy.)

(Note that powerpc-440fp is not a hw-target for powerpc, it's a different
processor instruction set like i586 vs i686. Alas, one that qemu doesn't
properly emulate yet.)

Now using last GPLv2 release of GCC, and future plans

Note that bash 2.05b is still intentionally ancient (to avoid the
bloat of 3.x), and may be replaced with busybox ash as the default FWL shell
in a future release. Similarly, although binutils 2.17 was the last
GPLv2 version from the FSF, an upgrade to
the
last GPLv2 release of Red Hat's binutils fork, 2.17.5.0.17 is under
consideration for a future release. But mostly the GNU compiler toolchain
has been upgraded as far as possible for the foreseeable future, either because
new versions provide no significant benefits or because they're not available
under a license compatible with the Linux kernel's.

Future upgrades to these packages will either be via individual patches
(maintaining GPLv2 license) or by investigating alternative non-FSF
packages such as OpenBSD's PCC or
Apple's LLVM. In preparation for support of
alternate compilers, the build scripts have moved from using "gcc" to the
more generic "cc" for the compiler name wherever possible, but currently no
drop-in replacement compiler is available.

Implementation details

The download.sh stage now populates a "packages" directory instead of
"sources/packages", meaning that everything in the sources directory now
comes from the FWL source tarball. To do the equivalent of "make clean"
you can "rm -rf build", and to do a "make distclean" you can
"rm -rf packages build".

The build no longer deletes the alt-* tarballs when UNSTABLE isn't
set, you must do that by hand if you want an updated alt- version of a
package. The packages/MANIFEST file creation can now use the extended
tarball information to detect git version numbers.

The === bars now show the current stage name and architecture each
package is being built for. (Pipe the build output to grep "^===" to see
just the start of each new package build.) The same information is sent
to the title bar of your xterm, export NO_TITLE_BAR=1 if you don't want the
build to update the title bar.

The BUILD_STATIC option now applies to all packages built in
root-filesystem.sh, and can be set from a target configuration's settings file
(as sparc is now doing).

The BUILD_VERBOSE option passes V=1 to the linux, busybox, and uClibc
builds that otherwise sanitize their output.

You can now set NATIVE_TOOLCHAIN=only to build just the compiler
and not the other parts (busybox, make, toybox, bash...) This is used
by the canadian cross steps in build.sh to create static cross and native
compilers.

This release broke up scripts/include.sh so that script just
sets lots of environment variables, and now includes scripts/functions.sh
to define shell functions. The architecture setup (reading architecture
files and creating temporary directories and such) is now done via the
"read_arch_dir $ARCH" shell function, so merely including sources/include.sh
should no longer have significant side effects. Similarly, setupfor now
has an EXTRACT_ONLY environment variable instead of depending on a blank
$ARCH (which host-tools.sh has, despite wanting to snapshot source so it can
build stuff).

Various fixes to make parallel builds work better. (So it neither leaves
background processes running nor kills the parent shell and closes your
xterm when you hit ctrl-c.) The new "killtree" shell function,
which kills a process and all its children and grandchildren recursively with
pgrep. We also build our own sources/toys/mysetsid.c program with the host
compiler (because the normal command line "setsid" doesn't also do a tcsetpgrp
on stdin so signal handling isn't forwarded to the new session). This should
more reliably allow ctrl-c to exit smoketest.sh and system-image.sh and such,
and let FORK=1 ./buildall.sh stop all its background processes when
interrupted. (Note that smoketest-all.sh also honors FORK=1.)

The logs from buildall.sh now live in build/logs.

The new config option ROOT_NODIRS avoids creating the normal set of empty
directories in the new root filesystem. (The old /tools support went
away, but a BUILD_STATIC ROOT_NODIRS build can be extracted into /tools
and used from there. Then either symlink "/lib" to "/tools/lib" or "export
UCLIBC_DYNAMIC_LINKER=/tools/lib/ld-uClibc.so.0" to add more stuff to
/tools natively.)

The compiler wrapper (sources/toys/ccwrap.c) can now use
architecture-specific WRAPPER_TOPDIR values. (I.E. armv4l-cc could
"export armv4l_WRAPPER_TOPDIR=/path/to/directory".) This makes
canadian cross compiles a lot easier.

April 2, 2009

Version 0.9.6 includes
Linux 2.6.29 and uClibc 0.9.30.1, now with support for sh4 and upgraded powerpc
support.

The big news is the Gentoo
From Scratch project, which extends a FWL system image into a Gentoo
Stage 1 environment, building natively inside qemu.

The documentation has been extensively
rewritten (but needs more). The source now contains a README and each image
has a usr/src/MANIFEST file listing the packages it built from.

The cross and native toolchains now include ldd, readelf, and
ldconfig. Added support for hardware targets (an unfinished example
of which is hw-wrt610n) that use the same cross compiler and system
image as a qemu platform, but build/package a different kernel to boot
on actual hardware.

Several new build options:

$SYSIMAGE_TYPE defaults to "ext2" but can now
produce and boot "initramfs" images as well. (And squashfs, but that doesn't
work yet due to missing features in squashfs tools version 3.4.)

Setting $HOST_BUILD_EXTRA compiles qemu 0.10.1 in host-tools.sh.
You currently need to do this to run sh4 or powerpc, 0.10.1 has some bugs
needing patches. Not on by default because it takes a _long_ time.

$NO_CLEANUP keeps source around after builds for debugging purposes.

$SYSIMAGE_HDA_MEGS lets you control the size of the ext2 system images.

$SNAPSHOT_SYMLINK lets the extracted source live on a different
filesystem than the build happens on. (Once the updated squashfs userspace
tools ship, build-static-toolchains.sh may be updated to use this.)

$CROSS_SMOKE_TEST use qemu application emulation to test run hello
world program at the end of the cross compiler build stage.

$SKIP_STAGE_TARBALLS don't create tarballs of each stage,
just populate the directories.

The run-emulator.sh scripts of system images have several new options,
do "./run-emulator.sh --help" for a list. Takes several environment
variables such as $QEMU_EXTRA and $KERNEL_EXTRA. Root filesystems can
now run in read only mode with "KERNEL_EXTRA=ro".

Lots of build infrastructure upgrades. The build stages have been
reorganized somewhat (most noticeably package-mini-native.sh is now called
system-image.sh). The "sources/packages" directory has moved up a
directory (to separate repository files from stuff downloaded at runtime).
Upgrades to the cc wrapper and the USE_UNSTABLE infrastructure. Better
stripping of the cross compilers and system images, so everything's a bit
smaller. The mirror list now lives in download.sh. New buildall.sh is a
wrapper around build.sh. New build-static-toolchains.sh builds them under
qemu; set FORK=1 to build them in parallel just like buildall.sh. The
ext2 system images are now built at a small size with genext2fs and
then expanded with resize2fs (because genext2fs slows down exponentially
as size increases; this means e2fsprogs is built in host-tools.sh now).
Distcc has been upgraded, and should work more reliably now.

Too many little bug fixes to list.

December 16, 2008

The mailing list moved to firmware at impactlinux.com. The link in
the nav bar on the left has changed, but the
old
archives are still there for the moment.

December 3, 2008

After several false starts (I.E. last minute bugs, yes I test this thing
before shipping it),
Version 0.9.5 is out, inching
towards a 1.0 release. Lots of new work this time.

The biggest news is uClibc 0.9.30 and uClibc++ 0.2.2. The native toolchain
now supports C++, hooked up to distcc and everything.

The documentation has been extensively
rewritten. (There's always more to do, but most of it isn't horribly out
of date anymore.)

The packaging stage now uses genext2fs instead of User Mode Linux, so
the most brittle part of the build has been removed. FWL can now fully
rebuild itself under itself, all the way through the packaging stage.

My personal website's getting a touch overwhelmed, so a higher bandwidth
site is now available at
impactlinux.com. This should become
the new primary website going forward.

BusyBox has also been upgraded (to 1.13.0), and the new USE_UNSTABLE
infrastructure allows test versions of any package to be downloaded and built
without forking the build scripts, so future package upgrades can be dealt
with as side projects without blocking other important work.

The packaging stage now uses genext2fs instead of User Mode Linux, so
the most brittle part of the build has been removed. FWL can now fully
rebuild itself under itself, all the way through the packaging stage.

The old forkbomb script has been replaced with sources/build-all-targets.sh.
with an argument --fork to build all targets in parallel, and "--fork 3"
to limit the parallelism. This script can run from a cron job, producing
a "buildall" directory with tarballs, compressed logs, and a README file
listing the version of each package. The environment variable USE_STATIC_HOST
specifies an $ARCH to create statically linked cross compilers
for. (The build will then try to use them for the later stages,
and won't get far if you specify an arch that can't run on the host.)

The include.sh script has been broken up, with a new sources/functions.sh
containing shell functions (like download, setupfor, and cleanup) and
a new "config" setting the user-editable environment variables that control
the build. Several of these variables have been renamed and their number has
grown. You can now specify the colors for each build stage using the variables
at the end of config.

Building mini-native in a Linux From Scratch style /tools directory is
no longer the default. Instead set NATIVE_TOOLSDIR=1 if you want that
kind of layout instead of a more conventional layout. (Similarly,
NATIVE_TOOLCHAIN=1 now enables the native toolchain, but that variable
defaults to enabled in config.)

August 6, 2008

Version 0.9.0 is out
(changeset 378)
and can rebuild itself under itself. (The packaging step still requires User
Mode Linux to create ext2 images, which only works on x86 and x86-64 hosts.
The next release should replace that with something more portable.)
The root filesystem is now based on BusyBox 1.11.1 (plus a few bug fixes).

The prebuilt binary images have been moved into their own
binaries directory.
The prebuilt binary cross
compilers are statically linked against uClibc for better portability.
Updated the README to better describe the
contents of the system-image
tarballs.

The forkbomb.sh script can now take a number of parallel targets to
launch, for example "./forkbomb.sh --fork 3". (The resulting display is kind
of horrible but the out-$ARCH.txt files are still legible.) Leaving off the
number still builds all targets in parallel, with the original display.

The include.sh script was reorganized so that interesting environment
variables the user might want to export for themselves are set near the top
of the script. Currently this includes:

BUILD_SHORT - Makes the mini-native.sh stage skip building
development tools. The resulting mini-native root filesystem contains uClibc
and busybox, but nothing else. (Export "BUILD_SHORT=headers" to put back the
toolchain headers for use by another compiler, such as tinycc.)

BUILD_STATIC - Tell cross-compiler.sh to create static
binaries, for a more portable cross compiler toolchain. (If you want them
linked against uClibc, build a mini-native for your host, chroot into it, and
then rebuild under that.)

PREFERRED_MIRROR - Tell download.sh to fetch packages from a
non-default location (falling back to the standard mirror list if it can't get
one from there).

RECORD_COMMANDS - Record a list of all command lines used during
each build stage log files named "cmdlines.$STAGE.$PACKAGE" under the build
directory. Each file can be turned into a list of the command names used via
"awk '{print $1}' build/cmdlines* | sort -u". The script
sources/toys/report_recorded_commands.sh gives a report of all commands
used by each package after a call to "RECORD_COMMANDS=1 ./forkbomb.sh".
(This is an easy way to find out exactly what environmental dependencies a
package needs to build, or at least what it uses out of $PATH).

Updated documentation is in progress...

June 6, 2008

The distcc trick accelerates a native build by calling out to the
cross compiler. To use it, download the appropriate cross-compiler-$ARCH and
system-image-$ARCH tarballs, run the "run-with-distcc.sh" from the system image
directory with the path to the cross compiler directory as its first argument.
(You need to have distcc installed on the host system.)

After building from source, the cross-compiler and system-image directories
remain in the build directory. The "emulator-build.sh" runs these with
distcc acceration set up. The script "smoketest.sh" is a wrapper
around emulator-build.sh which feeds a script into qemu's stdin to compile
and run a multi-threaded "hello world" under the emulator, using distcc and
the cross compiler.

If smoketest.sh works for a platform, it means a lot of stuff worked to get
that far. The cross compiler worked to build the system, and the emulated
system booted so QEMU, the uClibc config, and the kernel config agreed.
The emulated system has a working virtual hard drive and serial port, and for
distcc to work it has a working virtual network connection. Running "make"
would also require a working realtime clock.

The seven commands missing in order to run the FWL build under itself
are bzip2, sort, diff, wget, install, od, and find. (The busybox versions,
where available, had a bug, and they're not in toybox yet.) I'm trying to
get those in for the next release in September.

Sorry I missed a release back in April. Shouldn't happen again.

January 29, 2008

Version 0.3.1 is out
(changeset 275), with
kernel 2.6.24. The images
are now tarballs each containing the ext2, zImage, and run script files.
The run scripts now run qemu-setup.sh by default so /proc, /sys, /dev
and the virtual network are initialized, and the default shell is busybox
ash (so cursor keys and history work).

Sparc is still broken (tracked it down to a dynamic linking bug, I think,
haven't fixed it yet) and m68k is failing to build with an internal compiler
error (next release I should upgrade the compiler). All the other targets
have a working virtual network, and all but powerpc exit qemu when you
exit the PID 1 command shell.

Building on an x86-64 host< should no longer be insanely slow (the
gcc-min-heapsize arguments that allow 32-bit hosts to build in
128 megs of ram do very impolite things to 64-bit hosts), and the resulting
x86-64 cross compiler should now be properly relocatable.

Better build dependency tracking: the host-tools script is now
populating a single directory with all the commands needed by the build, and
restricting $PATH to that directory (so it can't accidentally call any
commands that aren't explicitly listed). It's building toybox and busybox to
provide most of these commands (the remaining ones are listed towards the end
of include.sh), and creating symlinks to the remaining ones out
of the original $PATH. This allows it to build on a wider range of host
distributions.

November 2, 2007

Version 0.3.0 is out
(changeset 242), with kernel 2.6.23. PowerPC finally works under
qemu (network and everything), and it's building a m68k target
(although I have no emulator to run that). That means there are
ten targets, of which eight run under qemu. (Sparc is still
broken, let me know if anybody actually cares.)

The new "./forkbomb.sh --fork" script builds all the targets in
parallel, or use --nofork if you haven't got the memory for that
(and --watch to watch and existing build if you ctrl-c out of
the display before it's done).

The build is now smarter about extracting only one copy of the
source code (in build/sources) and re-using it for multiple builds,
building out of tree (via cp -rs to create a tree full of symlinks
so the package doesn't need native out-of-tree support).

September 2, 2007

Added a porting guide. I'm working on
m68k, ppc, big endian arm, and maybe blackfin targets for the next release,
and thought other people might be interested in the process. (The vast
majority of it is fiddly debugging. These are the other steps.)

June 27, 2007

Version 0.2.2 is out
(changeset 185). Using uClibc 0.9.29 release, several small bugfixes,
new mips big-endian config, actually work when /bin/sh->dash (mostly
by explicitly saying /bin/bash everywhere), automatically detect
the number of CPUs in the machine to feed -j to make, and better
diagnostic output.

April 9, 2007

Version 0.2.1 is out
(changeset 156). Significant upgrades to armv4l: the native compiler works
now, and it's using soft-float. Added armv5l and i586 targets to show how
to do that kind of variant. Added preliminary powerpc support (although
qemu can't run it yet). Reorganized the download directory and rewrote
the README. Added squashfs back (the kernel
supports it, but it's not using it yet). Moved uClibc to miniconfig.
(Yes, the patch for this is huge, and yes I need to push it all upstream into
the Linux kernel. On the bright side, this one builds on a host that doesn't
have curses installed.) Stopped trying to build qemu (at least until they
get the gcc 3.x dependencies cleaned out), now just warn if it's not there
(so far the build only uses it to sanity-test the cross compiler, which is
now skipped automatically if it's not there).

I note that some versions of Ubuntu (Edgy and up) are pushing a political
agenda. They install bash, but point /bin/sh to dash instead. (So why
install bash if you're not going to use it?) This breaks all sorts of things,
and if somebody can tell me how to force make to run "/bin/bash" instead of
"/bin/sh", I'll consider trying to deal with the mess. Until then, I suggest
"rm /bin/sh; ln -s bash /bin/sh" to get a working system. (I'm no fan of
bash, but dash is just broken.)

(Those of you who use vi and want to be able to use the cursor keys while
in insert mode might want to do something similar with vi->vim. Ubuntu seems
to be trying to train users to change their habits. Dog biscuits would be a
more honest approach.)

Update: I've been pointed at
this as evidence Ubuntu isn't
pushing a political agenda. Ok, maybe it's just really bad technical judgement.
Are they honestly saying that people write shell scripts for the speed? And
how do you make a size argument and then install bash by default in ADDITION
to the other shell? Boggle. In 1991 after Linus Torvalds taught his term
program to understand the minix filesystem so he could upload and download
without rebooting, he taught it to handle the system calls of bash so he could
rm/mv/mkdir without rebooting. That's where Linux 0.0.1
came from. Bash was the default Linux shell _before_ 0.0.1. Be very
careful messing with that kind of a de-facto standard, guys. Dash is
garbage. Background a process with & and then hit ctrl-c: it'll kill it.
It doesn't understand blah/{blah,blah} file list syntax. It can't do
"source file.sh" (just ". file.sh", which is unreadable). I repeat: I'm not a
fan of bash (yeah, the bloat's getting crazy), but dash is broken.

And that still doesn't explain why "vi" can't use the cursor keys in
insert mode, but "vim" can...

March 6, 2007

Version 0.2.0 is out. This one builds ext2 images which can boot up to a
shell prompt under qemu. This works on i686, armv4l, x86_64, and mipsel.
(Sparc's broken. Anyone who cares about sparc is welcome to debug it.)

There are now prebuilt ext2 image files in the download directory.
The kernels are in the mini-native tarballs. (Yeah, I should repackage that
next release.) The system emulator invocation for each platform is
the appropriate sources/configs file, and run-mini-native.sh invokes 'em.

The mailing list is in the usual place if you have any questions.

January 14, 2007

There's a new irc channel for the project, #firmware on freenode.

Changeset 88 builds
i686, x86_64, armv4l, mips, and sparc. I've made a release tarball of
that (firmware 0.1.1, "It works
for me"), and updated the prebuilt cross-compiler tarballs in the
downloads directory. I've added a big README and
prebuilt mini-native tarballs for each platform (although packing them up into
something qemu can boot is currently left as an exercise for the reader).

Speaking of which, I'm currently working on adding ext2 packaging
(via toybox) to the build scripts,
so qemu system emulation can boot the result and then run the next stage
automatically.

December 28, 2006

Changeset 68 builds
a native build environment with a working toolchain. And in celebration, I've
revamped the website with an actual navigation bar and content and such.

I should put out a release soon.

December 5, 2006

Changeset 27 builds
a relocatable armv4l cross-compile toolchain! Download the tarball,
run ./download.sh, then run ./build.sh, then grab the "build/cross-compiler"
directory and use "bin/armv4l-unknown-linux-gnu-gcc" out of that (which
I admit is a bit of a mouthful, for which I blame the FSF).