cowbuilder is a nice tool which builds packages (optionally in parallel) in clean chroots quickly by making use of copy-on-write technology. This document will describe how to set up your Debian/m68k ARAnyM to use an already prepared base.cow until such time as an updated Aranym/Quick image has been uploaded. (This should not take too long any more.)

Set up the repository

Make sure you have the following two lines in your /etc/apt/sources.list:

If you come from Aranym/Quick

Use apt-get --purge --no-install-recommends install FOO generally. Add the two keyring packages and openssh-server so you can ssh in instead of using the console.

You can add --no-upgrade to the apt-get command line to prevent more changes to the host OS than necessary. (Or you can just try and a-g d-u the host system too. Takes some hours…)

Install the necessary packages

You will need to install at least cowbuilder and a TLS capable kernel image of your choice (the ones in Debian include the modules, so even if you use Aranym which has the kernels “outside”, like classical Xen without pygrub, you must install a kernel image in the m68k VM for the LKMs to be available. An example image is linux-image-2.6.38-2-atari (which the author of this article uses on Aranym). You also do want to install debian-archive-keyring and debian-ports-archive-keyring (re-run sudo apt-get update to get rid of “WARNING: the following packages cannot be authenticated!” style messages).

It’s also good to install eatmydata as it yields a (small, but on m68k we’re thankful for every bit) speed-up (whose cost is the potential loss of data integrity, but as the chroots are thrown away after building, this doesn’t matter).

You also need tar and gzip (which you have anyway) to extract the tarball.

The file /etc/pbuilderrc wants love, unfortunately the defaults are not very good. Here’s one you can use (it’s basically a POSIX shell script, as to its syntax) and customise:

# to have a different shell prompt inside the chroot
export debian_chroot="pbuild$$"
# set this to your favourite Debian ports mirror
MIRRORSITE=http://ftp.ports.debian.org/debian-ports/
# desirable, unless you *want* to run most time-consuming
# testsuites, which usually fail anyway… (yay for optimism)
export DEB_BUILD_OPTIONS='nobench nocheck'
# either disable ccache…
CCACHEDIR=""
# … or enable it; yes, all of this is needed, and do ensure
# the directory and all of its content is writable by the
# pbuilder user
#CCACHEDIR="/var/cache/pbuilder/ccache"
#export CCACHE_DIR="/var/cache/pbuilder/ccache"
#export PATH="/usr/lib/ccache:${PATH}"
# which dependency resolver to use? default is aptitude,
# a package in very bad shape in m68k (but not rebuilt
# due to its heavy dependencies); it’s not usable atm.
# uncomment this to use one internal to pbuilder (shell script):
PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-classic"
# various settings
export DEBIAN_FRONTEND=noninteractive
#export ARCHITECTURE=$(dpkg --print-architecture)
export ARCHITECTURE=m68k
export PKGNAME_LOGFILE_EXTENTION=_${ARCHITECTURE}.build
export PKGNAME_LOGFILE=yes
#REMOVEPACKAGES="debconf-i18n"
export LC_ALL=C

Set up the cowbuilder “base” chroot

Download the base.cow tarball…

What’s in it?

I have debootstrapped this using cowbuilder with the three keyring packages (Debian, Debian ports, my repo) as well as the helpers’ in-chroot parts eatmydata and fakeroot, plus pbuilder’s helpers apt, ccache (see below). Then I set up its etc/apt/sources.list and etc/apt/apt.conf, purged everything not in Debian proper (i.e. the keyring part) and otherwise obsolete (atari and mac fdisk, atari bootstrap), upgraded it, copied it to my work system, cleaned it up, made a tarball. So it contains only Debian/m68k material and configuration files.

If you enabled/disabled ccache or switch the dependency resolver in /etc/pbuilderrc (or ~/.pbuilderrc of course), or change other settings such as EXTRAPACKAGES and REMOVEPACKAGES, you must run sudo cowbuilder --update (or sudo eatmydata cowbuilder --update which is perfectly safe) afterwards. This gives pbuilder a chance to install/purge its helper packages. The update will also run an apt-get update and upgrade inside the chroot. Since I omitted the lists and package database from the tarball, it’s always needed at least once after extraction.

… or make your own

At the moment, Debian/m68k requires packages from both the unstable and unreleased suites (we hope to be able to get rid of using unreleased again, eventually, but due to the squeeze freeze, bugs, slowness, etc. we’re not quite there yet). However, debootstrap, which is used by cowbuilder, cannot pull from more than one repository. So, you’d need to use an inofficial one instead (which also involves installing its keyring package into the host OS first, etc). Really, save yourself the hassle and use the pre-made tarball.

Also, Debian unstable is a moving target. I actually failed the first time trying to bootstrap, due to some dependency issue… once you have it done, upgrading can be partial (with packages held back until the architecture caught up) and things continue to work.

Do not bother with debuild/pdebuild, it’s useless (it first creates a .dsc, then runs pbuilder on that, instead of building directly from the unpacked source package). Instead, just create your *.dsc as usual (usually dpkg-buildpackage -rfakeroot -S or something like that) and proceed as above.

Pick up the results

Look in /var/cache/pbuilder/result/ for your *.changes files and the ones listed in it. There will also be *_m68k.build files containing the cowbuilder log.

Run debsign -k01234567 *.changes or debrsign -k01234567 user@host.domain *.changes afterwards, where 01234567 is your PGP Key ID, to get the *.changes and *.dsc, if any (not -b/-B), files signed. (Note that some debrsign versions may support only one file argument.)

You can upload to Debian-Ports using dput with this in ~/.dput.cf using dput dpo *.changes style commands:

Advanced techniques

ccache

IMPORTANT when using ccache: If the toolchain (binutils, gcc, …) changes, you should reset the cache. It can only be done when no builds are running (which also gives you a chance to remove stale /var/cache/pbuilder/build/cow.* directories). This should work: sudo -u pbuilder env CCACHE_DIR=/var/cache/pbuilder/ccache /usr/bin/ccache -s

However, I could never get it to actually useccache during compilation, even with the three variables (CCACHEDIR, export CCACHE_DIR, change PATH) set… maybe you ❦ have more luck and document that here ☺

When doing porter uploads…

You do not want to spam people. Add export DEBEMAIL='My Name <mylogin@debian.org>' to ~/.profile, and every time you build a package for uploading (Porter NMU), use a command line like this (as it’s a porter NMU, it targets unstable, and thus you only want the binary-arch part):

Afterwards, add --hookdir $HOME/pb-hook to all cowbuilder invocations. This is nice to inspect the chroot to find out just why a build may have failed. (I’m not sure you can just chroot into /var/cache/pbuilder/build/cow.〈PID〉 or whether that may circumvent cowdancer, so don’t do it, just run on that shell.) Once you leave it, the chroot will be cleaned up.

Wrapper scripts

These must be chmod +x’d, of course.

Some explanations: * Resetting the locale, because sometimes, some things get passed from outside the chroot to the inside, yielding weird error messages about locales and other things. Probably should use env -i and a whitelist of allowed environment variables, which would also make stuff more deterministic (think of ${DEB_*} flags), but hinder easy customisation (not a problem though…). * Checking whether $DEBEMAIL is of the form User Name <login@debian.org> since we assume that when passing it to dpkg-buildpackage’s -m option – $DEBEMAIL has other valid forms, notably just the eMail address, but then you have to set other variables, so I never bothered and ever only set this one. * Feel free to improve upon those ☻

speed up

While no cowbuilder/pbuilder is running (the directory /var/cache/pbuilder/build is empty), you can edit files inside /var/cache/pbuilder/base.cow to your heart’s desire. The etc/apt/apt.conf and etc/apt/sources.list have already been customised; if you want, here’s some more:

prevent in-chroot man-db from updating its cache

force specific versions of packages to be installed

… for example, when we have a too-new arch:all package depending on an arch-dep package that’s not built yet, and we don’t want to recompile everything just to get this working. Note: wildcards are not supported, contrary to the pinning documentation, prior to, IIRC, version 0.8.14.1 of apt.

sudo $EDITOR /var/cache/pbuilder/base.cow/etc/apt/preferences

share APT package cache with host system

Do not run apt-get autoclean when you do this. Manual, if any, cleanup required.

Then, append the following line (yes, that’s the apt.conf outside the chroot):

Dir::Cache::Archives "/var/cache/pbuilder/aptcache";

We keep the symlink just to stay out of trouble.

multi-distribution/-suite pbuilderrc

Not just for m68k but in general (not really useful on m68k). Can also build i386 packages on an amd64 system. Is multi-tenant capable. CVSweb link to latest version – Remember that packages uploaded to Debian proper MUST be built on bare metal, not in an emulated environment, except if the porter team for the target architecture allows otherwise (m68k explicitly allows ARAnyM, but e.g. ARM explicitly disallows emulated/virtualised environments)!