Component Compilation Notes

WARNING: This page is out-of-date. DIET-PC 3
compilation automation is now provided with the source code.

This file contains my instructions for compiling DIET-PC binaries. This is
a work in progress; if you are viewing a copy of this file included with a
DIET-PC source code bundle, please check the online version for updates.
Instructions provided are intended for DIET-PC 2; software versions actually
deployed in DIET-PC 1.1 may differ, and may require different workarounds or
configuration parameters when building.

You're no doubt wondering why I don't have a complete CVS source code tree
and a master Makefile that just builds everything. Well, there are two reasons
for this: (i) it's difficult, and (ii) it would slow the rate of change such
that DIET-PC would have to use older versions of most software components. As
you'll see below, DIET-PC builds of most software components are heavily
modified, by DIET-PC-specific patches (I try to get my patches rolled in to the
official releases in cases where the issue isn't DIET-PC-specific), or modified
build procedures to produce "embedded-friendly" binaries, or both. In many
cases, automation provided with the software distribution is too inaccurate for
this sort of work, and binaries need to be relinked by hand without bogus
dependencies introduced by "shotgun" automation. Needless to say, writing my
own automation to resolve all of these issues would be immensely time
consuming, so you're going to have to do some of the work manually instead.

The official build environment for DIET-PC is Debian: Woody (3.0) for
DIET-PC 1.1, Etch (4.0) for DIET-PC 2, and Lenny (5.0) for DIET-PC 3. If you
choose not to use Debian, pay particular attention to the glibc requirements
for each DIET-PC release: not greater than glibc 2.2.5 for DIET-PC 1.1, 2.3.6
for DIET-PC 2, and 2.7 for DIET-PC 3.

Ultimately I will ensure that all code compiles cleanly under gcc 4.x
(although I use g++ 2.x to produce smaller C++ binaries), but at present
gcc 3.x may still be required for some compilations.

One of the aims of DIET-PC is to be developer friendly; I don't want to
force developers to use a dedicated build environment if this can be avoided.
I try to keep DIET-PC reasonably up-to-date with current software versions
(even if this causes some bloat), so that straightforward transplantation of
binaries compiled on the likewise reasonably-up-to-date Linux distro that you
are hopefully using stands the best possible chance of success.

Nevertheless, it's likely that in some places my instructions are not going
to work for distros other than Debian. If you're reasonably familiar with
compilation tools you should be able to sort out any minor differences. Pay
close attention to shared library dependencies - DIET-PC does not have a full
range of shared libraries. If you end up with a binary that needs a library
that DIET-PC doesn't have (use "ldd" to show dependencies), either recompile it
without the feature that introduces that dependency (typically using a
--without-xyz or --disable-xyz GNU configure option), or if you can't do that,
link the binary manually with that particular library static, using
"-Wl,-Bstatic -llibrary -Wl,-Bdynamic". In some cases, such
as when a library is loaded by an application rather than via the dynamic
linker (eg. libdvdcss), it may prove necessary to compile and include that
shared library in your DIET-PC build. Beware of dynamic linkage of
libgcc_s.so.1 under some circumstances and with certain versions of gcc (I
want to avoid adding this library to DIET-PC (or DIET-PC 2, at least)). This
behaviour can be suppressed by adding the "-static-libgcc" option to
CFLAGS or similar.

Gentoo might not be a suitable choice for a development environment if you
intend to deploy to a diverse range of hardware. This is because any static
library that you link against is likely to have been compiled for the specific
instruction set of the Gentoo host's CPU (i.e. -march=xxx) and
consequently the code you produce will not run on older hardware.

It is assumed that ~/diet-pc-src contains all relevant downloadable
source code objects (tarballs, DIET-PC patches, config file templates). The
target directory tree is assumed to be located at ~/diet-pc. For
best performance, I recommend performing the actual compilation in a tmpfs
filesystem (/tmp or /dev/shm).

I have not provided URLs for most source code distributions, as this is too
much of a moving target. You can almost always find the source by doing a
project search at freshmeat.net. In cases
where you can't, I will endeavour to provide a URL.

The current user is assumed to be root, with a BASH shell. The shell's umask
is assumed to be 022. It isn't necessary to compile as root, it's
just more convenient for installing. If your distro's root shell environment
aliases the cp, mv and rm commands to force usage of
the "-i" (interactive) flag for safety, you you should remove these aliases
(eg. "unalias cp") before attempting to cut and paste multiple lines from this
webpage.

To avoid repeating myself, all of the instructions below now assume that
you have set CFLAGS and CXXFLAGS environment variables appropriately for
embedded compilation for your platform. The easiest way to do this is to add
the following to your ~/.bashrc file:

In comments in the build procedure, I use $CFLAGS as shorthand for what is
set above, even in contexts where the literal string "$CFLAGS" is
not understood. When editing files, substitute what $CFLAGS evaluates to (i.e.
the output of the shell command "echo $CFLAGS") rather than the
literal string "$CFLAGS". Failure to set $CFLAGS correctly will
not prevent compilation from succeeding, it will just result in a binary that
is suboptimal for DIET-PC use (-Os avoids optimisations that result in bigger
binaries; -mtune=XXX tries to make the code run most efficiently for the named
processor class while not excluding other processor types (which is what
-march=XXX does)).

Instructions assume Intel 80x86 architecture by default. Where the above
environment variable differences are not sufficient to cater to other platforms
and there is a platform-specific variation to the procedure, obey the comments
provided.

It shouldn't make any difference, but FYI, my ARM build platform is a
ThinLinx Hot-e HL200 (an ARM920T-based SBC), my PowerPC build platform is an
Aluminium G4 Powerbook (7400 series CPU), and my x86/x86_64 build platform is
an Intel Core 2 Duo 2.66 GHz based on an ASUS P5KC mainboard. It is possible
to use cross-compilers, but I have a strong preference for native or emulated
compilation, to avoid oversights and complications. Native compilation can be
emulated using QEMU ("integratorcp926" for ARM, "prep" for
PPC and "mips" for MIPSEL); if you have a fast x86 CPU this may even
be faster than using real hardware in some cases (eg. ARM and MIPSEL).

If you are compiling under Debian Etch, please install gawk and ensure that
/etc/alternatives/awk points to this. Mawk will not work with some of the
complex awk scripts that the glibc build automation uses.

Debian glibc patches can be found in the debian/pool/main/g/glibc/
subdirectory on any Debian mirror.

The DIET-PC GRUB build uses Debian patches, primarily for the splash graphic
feature. I have further modified GRUB with a patch by Sylvain Fourmanoit
sourced from a bugtracker on Savannah (possibly still available here) to add
a Vendor Class Identifier - "GRUBClient" - to its DHCP requests, so that you
can configure DHCP servers to limit or customise responses to GRUB clients.

The grub_netdriver_list.txt file can be found in the DIET-PC source code
tarball or separately here.

GRUB is specific to the x86 and x86_64 ports of DIET-PC. It is not 64-bit
safe and must be compiled 32-bit. Nor is it gcc4-safe, apparently!

These build instructions are for the generic_kernel_586+ package, which is
intended as a quick-start demonstration, not production use. You should treat
these instructions merely as a starting point for building your own custom
kernel, which you should install to ~/diet-pc/kernel, or some other directory
containing the word "kernel" that is distinct from
~/diet-pc/generic_kernel_586+.

We use this obsolete version of nbd because it creates a much smaller
nbd-server binary. Newer versions add features that we don't need, and link
against very large libraries, mostly for programmer convenience.

OpenChrome is a video driver for Via embedded chipsets that is more
capable and supports newer hardware than the Unichrome driver bundled with
Xorg. I use this because I happen to have one of these newer chipsets (CN700).
Use of this driver is entirely optional (if you don't use it, your
xserver_xorg_via package will be built using the bundled Xorg driver instead).

This driver will need access to your Xorg (xc) tree, so build Xorg first
(and don't clean up afterwards).

FIXME: This is the procedure for Xorg 6.9 only, need update for Xorg 7.x
using 0.2.902 official (Xorg 7.4).

OpenMotif is a (regrettable) necessity for the wfcmgr component of ICA
Client version 9.0 and later. There is no ICA Client 9 available for PowerPC
or ARM Linux, so at present there is no point in compiling OpenMotif for these
architectures.

OpenMotif's official homepage
is very outdated. For all intents and purposes, the real home
page is actually http://www.motifzone.com/. OpenMotif is shunned by many Linux distros
because of its license restrictions, but at present it is the only free
implementation that provides the full Motif 2.0 compliance that Linux ICA
Client 9 needs.

OpenMotif 2.2.4 contains important security fixes, but there isn't a
generic official source tarball, so we'll have to extract it from one of the
distro-specific SRPMs instead.

Ensure that you have xbitmaps and all of the xorg-dev group installed before
trying to compile this.

Your host platform no doubt provides OpenSSL libraries that you can link
against. However, there are some advantages in building a custom version.
Firstly, OpenSSL 0.9.7 is significantly smaller than OpenSSL 0.9.8. Secondly,
this version may be more up-to-date with regard to security fixes than your
host's. And thirdly, unlike your host's libraries, these will actually run on
an 80386 (because we remove "-m486").

This library isn't installed in DIET-PC directly, but it is linked against
by OpenSSH, Rdesktop, etc. Programs that require OpenSSL typically only use a
small subset of the library's functions, so unless you have several programs
that use encryption, it is more space efficient to link each such binary
against a static library.

The authoritative site for the pcimodules patch
(ftp://ftp.yggdrasil.com/pub/dist/device_control/pcimodules) seems to be
permanently unreachable, and copies can be difficult to locate. I have
updated the original patch (for pciutils 2.1.11) for pciutils 3.0.0; this
patch is available from the DIET-PC downloads area.

Version 2.2 is very outdated, but DIET-PC 2 must continue to use this,
because SquashFS 3.x is not backward compatible with 2.x, and the skeleton
package (which includes mksquashfs) must remain interoperable with the
earliest DIET-PC 2.0 kernel, which used SquashFS 2.0.

This is the newer version of SquashFS, not compatible with SquashFS 2.2, and
not used by default. If you want to use SquashFS 3.x, change the
~/diet-pc/tools/mksquashfs symlink to point to mksquashfs3 instead of
mksquashfs2.

On Debian (other than amd64), try "apt-get install libsysfs-dev"
instead. On Debian amd64, you will need to compile and install your own
libsysfs with the "-fPIC" option as below, because you can't safely
link a dynamic library against a non-PIC static library on an x86_64 platform.

If you have DIET-PCs in multiple timezones, it may be more convenient to
pass a TZ kernel parameter via the boot loader (see tzset(3) manpage) than to
create separate boot images with different /etc/localtime files.

Ignore the udev distribution's claims of a minimum kernel version of 2.6.15,
demands to disable /sbin/hotplug, and insistance that test-udev is "only for testing". In practice, udev will still
work satisfactorily with older 2.6 kernels as long as test-udev
is installed as /sbin/udev and /bin/hotplug still calls it.

It boggles the mind that the 2.6 kernel was released without stable
userspace tools, and worse, that the udev developers seem to have no
intention of ever providing stable releases, and change their minds like they
change their socks. The fact that the udev version number no longer starts
with zero does not imply that it is in any way "stable".

WU-FTPD no longer appears to be maintained (it had a history of security
problems, and was abandoned in favour of VSFTPD et al). Nevertheless it
remains the FTP server of choice for DIET-PC because of its FTP conversions
feature (on-the-fly compression and archiving). Note, however, that WU-FTPD
2.6.2 is not adequate for DIET-PC purposes - you will get segfaults.
Hence you should use a CVS snapshot available from
ftp://ftp.wu-ftpd.org/private/nocvs/wuftpd-cvs-current.tar.gz.

Note that this configuration is no good for compiling xclass applications
to run locally, since OX_DEFAULT_POOL and OX_DEFAULT_ROOT do not
match the prefix used. A prefix of /usr/local is only used here to avoid
installing software in /usr that is not under the control of your Linux
distro's package manager.

xclass is required to build x0rfbserver (rfb). Although you may use your
platform's xclass library (if available) instead of building one, this build
of the xclass library is optimised for space, to ensure a smaller x0rfbserver
binary.

FIXME: Need new entries for my own drop-in module-only solutions. This
solution will only create useable vnc.so modules for Xorg 6.9/7.0, or 7.2
using extra hacks. Still okay for Xvnc creation (although TightVNC code is
simpler for that).

N.B. Fixes 09 and 10 in the X336fixes.tar.gz bundle are unofficial and
DIET-PC-specific.

XFree86 3.3.6 won't compile for ARM - it looks like the ARM architecture
wasn't properly supported until XFree86 4.x. Nor will it compile for x86_64.
It doesn't seem terribly useful on PPC either, for that matter!

xine-lib is very awkward to compile for an embedded environment, because it
links to many back-end shared libraries using libtool. The back-end libraries
are often too big to include whole in an embedded environment, and we would
prefer to link some of these libraries statically into Xine's loadable modules.
Unfortunately libtool makes this very difficult.

Because of the (un)subtleties of libtool and because xine-lib needs to be
installed in order to compile xine-ui, we need to install xine-lib to /opt/xine
on your development platform. It's unlikely that this will clash with an
already installed package, but it would pay to check first.

Unfortunately the instructions below are likely to be somewhat
Debian-specific. Your distro's developer libraries might be static, shared or
(hopefully in most cases) both, and may or may not be cross-linked to certain
other libraries, and any number of development packages that xine-lib regards
as optional (but which are required by my particular built) may be absent.
Hence the only way to ensure a known state is to use the official DIET-PC
development environment virtual machine for your platform.

For full functionality, you will need to install a lot of
development packages (DEB, RPM, etc) on your development platform before
compiling xine-lib. Probably the best way to find out what you are missing is
simply to run the configure command and see what errors are reported (important
bits will be heavily asterisked). You don't need to install
everything that configure complains about. My build didn't include:
(a) pointless ASCII art libraries (AALib, CACA), (b) libraries irrelevant to
the Linux x86 platform (Libstk, DirectX, IRIX AL), (c) libraries that are only
provided in shared form (JACK), and (d) libraries that have too many further
dependencies to be viable in an embedded context (WAND, directfb, polypaudio,
gnome_vfs, libsmbclient, SDL, freetype and modplug).

These intructions assume that your static X libraries are in /usr/lib, which
is usual for X11R7. If you have an older X11R6-based distro, you may need to
change some X library pathnames to use the /usr/X11R6/lib prefix instead.

Except in the case of freetype (which can be safely turned off using
--disable-freetype), the way I have avoided autodetection of unwanted
back-end capabilities was simply to uninstall the relevant *-dev DEBs / *-devel
RPMs.

Note that although these instructions will allow you to build Xine for
PowerPC, in all likelihood it won't work, because the PowerPC runtime linker
can't reliably combine static and shared components in a shared library (due
to use of 24-bit relative addressing in objects that are compiled without
-fpic, resulting in possible out-of-range jumps). Libtool warns us that doing
this isn't always portable, and indeed in this case it isn't. The same is
likely to be true for x86_64 (not attempted yet).

Use of CURL will blow out the size of the binary and introduce OpenSSL
dependencies, so I suggest removing the libcurl3-dev / curl-devel package so
that configure won't detect and use it.

Because xine-ui's installation is fairly complex (and because you might
like to use Xine on your development box also!), we will install xine-ui in
full to /opt/xine on your development platform, and then prepare the DIET-PC
mmedia_xine package from that.