DIY Linux Reference Build

GregSchafer

Permission is granted to copy, distribute, and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
copy of the license is available at
http://www.gnu.org/licenses/fdl.html.

This document may be copied and distributed in any medium, either
commercially or noncommercially, provided that the GNU Free
Documentation License (FDL), the copyright notices, and the license
notice saying the GNU FDL applies to the document are reproduced in all
copies, and that you add no other conditions whatsoever to those of the
GNU FDL.

All other trademarks and copyrights referred to are the property of
their respective owners.

Revision History

Revision CVS

24th April 2009

Please refer to the
CVS
ChangeLog for full revision history.
Note: Due to the dynamic nature of
the subject matter this document is forever evolving. As such, there
will never be an official "release" with a version number.

Abstract

This is an attempt to document a robust method of natively bootstrapping
a modern GNU/Linux system from source, with a strong focus on technical
correctness, testing, automation and package management.

1. Introduction

1.1. Preamble

This document is dedicated to those crazy "Do It Yourself" Linux
enthusiasts who would rather build their own GNU/Linux system from
source code than run a precompiled binary distro.

DIY Linux is not for the faint of heart. There are many drawbacks and
not everyone will have the ability or desire to do it. You don't need to
be an experienced programmer but you must have above average knowledge
of software building principles. If all you want is a GNU/Linux system
that simply "just works" then you would be far better off with a
precompiled binary distro prepared by professionals. Nevertheless, if
you have a legitimate reason to build your own GNU/Linux system
completely from source, have sufficient technical skill, are a bit of a
control freak, or you're the kind of person who simply must be in the
driver's seat when it comes to your Linux system, you'll find a high
quality build recipe here.

The intended audience is the more technically adept Linux user. Folks
new to the world of DIY Linux should first head over to the
Linux From Scratch
(LFS) project to learn the basics.

1.2. Goals

This "reference" build is intended to be a solid foundation for building
custom GNU/Linux systems. It's only a Reference Build so feel free to
add or remove packages and customize it in any way you see fit. However,
please keep in mind that the build recipe and package versions presented
here have been tested and are known to work, so if you deviate you'll be
on your own. Of course you're on your own anyway as this is, after all,
a "DIY" project.

The goals can be summarized as follows:

Build a reasonably up-to-date and usable base GNU/Linux system from
source.

The term "usable" implies use of Package Management (PM), because a
system without PM can be frustratingly difficult to maintain.
Additionally, if PM is used to create binary packages, it's possible
to take advantage of the "build once, deploy anywhere" philosophy
thus simplifying maintenance and administration of multiple systems.
Therefore this build recipe strives to accommodate general Package
Management principles. You'll find the overall build method and
individual build commands aiming to be "PM friendly" towards a
number of existing Package Managers (see below).

The build method used to bootstrap the system must be robust and
work from a virgin default development install of every major Linux
distro released within the last 5 years. Upgrading tools on the host
defeats the purpose and therefore does not qualify. The method must
also work when coming the other way ie: from a host running a latest
cutting edge distro.

The base system must be able to build a wide range of commonly used
open source software.

The base system must be self-hosting i.e. be able to rebuild itself
reproducibly with verification provided by a binary comparison
technique known as Iterative Comparison Analysis (ICA).

Will use latest, stable, released, pristine upstream sources with as
few patches as possible and as minimal build commands as efficiently
possible (within reason). So called "beta" or "unstable" releases
will be used in exceptional circumstances (see below).

New technologies will not be adopted into the Reference Build before
they are ready.

1.3. Build Methods

The basic theory of operation involves building the target system
natively inside a chroot environment. To this end, two separate build
phases are utilized. The goal of the first phase is to build a
self-hosted temporary toolchain which is installed into a non-standard
prefix. The second phase is then conducted inside a chroot environment
whereby the result of the first phase is used as a platform for building
the final system.

There are currently two variations of the build method presented here.
The first is the so-called "Next Generation" method which leverages
cross compilation for the initial "Pass 1" toolchain. The second is the
original method which is now deprecated and no longer recommended. The
new method has 2 main advantages, 1) there is much less reliance on the
host system and 2) it is much better equipped to handle modern
"multi-arch" platforms like x86_64. For example, when targeting x86_64
it doesn't matter whether the host system is 32-bit or 64-bit or a
combination of both (userland and/or kernel), or whether you want a pure
64-bit system or a bi-arch (multilib) system. The Next Generation build
method caters for all these possibilities.

1.4. Architectures

The following architectures are supported by the Reference Build: i386
(x86), ppc
(PowerPC) and
x86_64 (AMD64).
We don't have access to hardware for other architectures therefore we
are unable to test or support them (this includes 64-bit PowerPC).
Please note the PowerPC and AMD64 builds appear to function perfectly
but they are not as well tested as the x86 build.

Important

The Reference Build assumes a mostly native build
environment. However, it should be noted that the Next Generation
build method employs cross compilation for the initial pass 1
toolchain only. This has many advantages. For example, if you are
running a 32-bit x86 system on AMD64 hardware and want to build a
64-bit environment (single or multi-arch), the cross toolchain can be
used to cross compile a 64-bit kernel at the appropriate point in the
procedure. The original build method does not support bi-arch and in
the case of pure x86_64, it required the host machine to already be
running a 64-bit kernel and have a 64-bit userland available.

1.5. Automation

It is assumed the reader will be scripting his/her own builds. In order
to make the script writer's life easier we have structured this document
in such a way as to allow direct extraction of (a) each package's build
commands ("scriptlets") and (b) overall package information
("packagedata"). You can find a tarball containing these files right
here. These scriptlets (and
packagedata) are extremely handy for integration into your own build
scripts.

A sample implementation of this "Document Centric Automation" is the
author's own build scripts which you can find
here.
It's worth noting that the author runs a full test build using these
scripts before any changes are made to the published version of this
document. This goes a long way towards ensuring that the build commands
as printed here are of high quality and free of typographical errors.

1.6. Build Notes

In addition to the goals listed above, the aim is to produce an
NPTL-enabled system based on:

Latest 2.6.X kernel.

Latest GCC releases. (Note:
Multiple versions of all major Toolchain components are supported.
Please see the important notes below
for details.)

Glibc development moves quite rapidly these days. While this is very
much welcomed, there is a downside in that the release schedules of
other toolchain components can sometimes get out of sync. This is a
problem because history has shown that you cannot easily mix and match
versions of toolchain components. In particular, Glibc compilation is
very sensitive to the GCC version. For example, you cannot expect to
compile a recent Glibc with a GCC from the 2.95.X era. It simply
doesn't work. In our experience the most stable and trouble-free
combinations of toolchain components are those of same or similar
vintage. Another point to consider, every new major GCC release is
stricter than the last. This means that once you get beyond the base
Reference Build you will almost certainly come across packages that
fail to compile with the latest GCC. For these reasons we have decided
to support multiple major versions of toolchain components. The DIY
build recipe has been carefully crafted to remain "out of the box"
compatible with multiple GCC, Glibc and Binutils versions as per the
table below. Simply set the corresponding environment variable for
each toolchain component (detailed in Section 2.1, “Environment”)
to build with your chosen versions.

Another important factor in building toolchains is which set of kernel
headers to use. The Linux-Libc-Headers package (LLH) worked well for a
number of years. But it has now been superceded by a new method known
as "make headers_install" (HDRS_INST) which was introduced in
linux-2.6.18. HDRS_INST is now recommended because it provides a
consistent and up-to-date set of headers that have the backing of
kernel developers, system library developers and distributors alike.

The table below shows some recommended toolchain combinations. Other
combinations might work but do not receive any testing, and in some
cases require patches (not provided here). If you decide to mix and
match versions, you need to be aware of the risks associated with
doing so. For example, occasionally you'll see warnings from upstream
developers like
this
and
this.

Glibc

GCC

Binutils

Headers

Comments

2.3.6

3.4.6

2.16.1 FSF

LLH

A classic. Rock solid, but starting to show its age.

2.6.1

4.2.4

2.18 FSF

HDRS_INST

Very stable. Was the previous default.

2.7

4.2.4

2.18 FSF

HDRS_INST

An excellent match. Current Reference Build default.

2.7

4.3.3

2.18 FSF

HDRS_INST

Seems to be stable. No known major problems.

2.8

4.3.3

2.18 FSF

HDRS_INST

Appears OK. But still quite new so tread carefully.

This table shows toolchain combinations that were once recommended but
are no longer tested. But in all likelihood they should still work
fine in the current Reference Build context.

Glibc

GCC

Binutils

Headers

2.3.6

4.0.4

2.16.1 FSF

LLH

2.4

4.1.2

2.17 FSF

LLH

2.5.1

4.1.2

2.17 FSF

HDRS_INST

Udev is not yet included in the Reference Build. It'll go in once (a)
there is unanimous Udev buy-in from all the major distros shipping 2.6
kernels which apparently hasn't happened yet and (b) Udev development
settles down to a point where upgrading to the latest version doesn't
break the system.

Bootscripts are not included in the Reference Build. Bootscripts tend to
be a matter of personal taste so folks are encouraged to write their own
or plug in an existing 3rd party package.

In the beginning we started out with something resembling the then
current LFS. We then developed modifications with a view towards
achieving the goals as listed above. Some other points worth noting:

The equivalent of LFS Chapters 5 & 6 are named the "Temptools
phase" and the "Chroot phase". Better names might possibly be
"Bootstrap phase" and "Sysroot phase" but confusion could arise as
those terms are already used in GCC parlance.

The Temptools phase is essentially all about bootstrapping a new
toolchain. Therefore it doesn't have to be perfect and it doesn't
demand that test suites be run. As long as it can build a verifiably
clean system (from a reproducibility point of view) inside the
Chroot phase it has done its job.

The pass1's of Binutils and GCC are no longer statically linked.
Static linking is a diversion from the real goal of the Temptools
phase and is a potential source of failure when what is needed is
robustness.

We have implemented use of `config.site' to increase efficiency with
the build commands. Every package that uses an Autoconf produced
configure script can take advantage of this. eg: specify
`--prefix=/usr' only once in config.site instead of multiple times
for every package.

We have dropped `make install' for many of the Temptools phase
packages to reduce cruft. A simple `cp -v <prog>
${TT_PFX}/bin' will often do the trick.

We have added the optional capability for parallel building on SMP.

We have added the optional capability to build without debugging
information via some careful use of CFLAGS and LDFLAGS. Stripping
the binaries will achieve a similar result but generating all that
debugging info in the first place is likely to slow down the build
(think disk I/O).

We have added `-v' to our invocations of the core file utilities to
enhance the level of system feedback while building.

We are compiling Glibc against sanitized kernel headers ie: the
`linux-libc-headers' package. Compiling Glibc against raw kernel
headers is always an option, but it goes against the Glibc
developer's general advice of recent times to use distro-supplied
sanitized headers. As of kernel-2.6, sanitized kernel headers are
the only realistic option for installation into /usr/include because
the raw headers are pretty much unusable when installed there.

The host MUST be running a 2.6.x kernel for the build of an
NPTL-enabled Glibc to succeed. If you're bootstrapping the build on
an old distro, you can usually get away with a procedure that
involves building a temporary non-modular 2.6.x kernel then booting
it with a grub boot floppy. See
here
for details. FIXME - must document this procedure properly.

1.7. Prerequisites

FIXME mention stuff like gawk for recent glibc, bison for bash, texinfo
for HJL binutils etc. Also point out how one can easily build "pass 0"
versions of these tools if getting them installed on the host is
impossible FIXME

1.8. Security

FIXME shoot down the hysteria surrounding security patches. It's horses
for courses. You want security? Then FFS run a distro which provides
security updates. Mention how utterly idiotic it is to build a system
yourself then place it in a security vulnerable situation (ie: on the
internet) without having the faintest clue about security then expecting
someone else to be responsible for patches. FIXME

1.9. Kernel

1.10. Package Management

There are many and varied techniques for implementing Package Management
(PM) strategies. This LFS
page
has a reasonable writeup of the various PM styles that exist today.
We've geared this Reference Build towards those Package Managers that
operate on the basis of diverting `make install' to a temporary location
for the purpose of creating an archive of the installed files. These
Package Managers typically use the commonly available Makefile variable
`DESTDIR' to achieve the diversion. Some examples of Package Managers in
this category are RPM (Red Hat/Fedora etc), Dpkg (Debian etc), Portage
(Gentoo), Pkgtool (Slackware), BPM (Bent Linux), Pacman (Arch Linux) and
Pkgutils (Crux). Please note, we don't insist that you employ PM when
using this Reference Build. All the build commands will still work
correctly if you choose to perform a "build in place" style installation
à la LFS. If you do decide to employ PM, feel free to choose any
Package Manager from the supported category that meets your needs,
because the Chroot phase installation commands are generic and should
work with all of them. In our experience Package Managers with simple
spec file formats are the easiest to integrate into a PM based Reference
Build. You could even write your own simple Package Manager if you felt
that way inclined.

In typical "build in place" style installations the entire Chroot phase
is performed as root. However, in most source building scenarios it's
generally considered best practice to build packages as a non-privileged
user, and only when it's time for `make install' should you need to
become root. Therefore in an ideal world involving PM, you would build
and create archives as a non-privileged user, then install the archives
with a Package Manager as root. We've set up this Reference Build in
such a way as to try and achieve that ideal world. Be aware that
creating package archives as a non-privileged user can sometimes result
in wrong file ownerships and permissions on files inside the built
archives. A potential solution to this problem is to make use of the
"Fakeroot" package which we've included as an optional extra. Please
note, we don't insist that you build packages in the Chroot phase as a
non-privileged user when using this Reference Build. Building as root
still works correctly.

2. Getting Started

2.1. Environment

The very first task is to set up a sane environment in which to perform
the build. These environmental settings apply to the initial Temptools
phase, though most will also be carried through into the Chroot phase.
But first, a lesson that many folks have learned the hard way:

Caution - Never build the Temptools phase as root!

There is a government advertising campaign here in Australia that goes
something like "If you drink and drive... you're a bloody idiot".
Well, the same thing applies to building the Temptools phase as root.
Never do it. The risk of trashing your host system is too great.

Create a specific user and group in which to perform the build of the
Temptools phase. Let's call them something creative like `build'. As
root:

Decide where you want to build and install the system to. This doesn't
have to be a partition, it can be anywhere, under $HOME, /mnt, /tmp or
wherever you like, as long as there is plenty of space there (FIXME how
much?). The location you've chosen to build the system will be assigned
to the $SYSROOT variable below. Once the build has finished, the entire
root file system image may be transferred to its intended final
destination.

The following steps are taken to ensure a sane shell environment in
which to work in. Your build script should, in practice, inherit these
settings but it would be wise to at least sanity check the variables
yourself when writing your script. Now login as user `build' and create
the needed bash startup files and config.site. Then source the files:

Important

Be sure to substitute the location you've chosen for the build in the
$SYSROOT variable assignment below. You'll likely also want to amend
the value of $TZ to match your local time zone. $DIY_TARGET is used to
select the target architecture you are building for. It is
vitally important to ensure that the
vendor field of the target triplet is something
other than "pc" or "unknown". This is
because we want to force the pass 1 toolchain into cross compilation
mode. For example, if your real target is i686-pc-linux-gnu, set
$DIY_TARGET to i686-diy-linux-gnu or even i686-FAKE-linux-gnu. If your
real target is x86_64-unknown-linux-gnu, set $DIY_TARGET to
x86_64-diy-linux-gnu, and so forth. Feel free to tweak $TT_PFX if you
don't care for the name "temptools". If you are building on SMP,
uncomment the MAKEFLAGS line to take advantage of make's parallel
execution feature. This is becoming more important as
multi-core
CPUs increase in popularity. Even on UP you can achieve quicker builds
with export MAKEFLAGS="-j2" (at the expense of
system responsiveness). If you are targeting x86_64 and also want the
ability to compile and run 32-bit code (recommended) set $BIARCH to
YES. This will provide a basic bi-arch (multilib) setup with 2
separate Glibc installations and the ability to compile the Grub
bootloader which depends on 32-bit compilation. From this basic
multilib setup, you then have the capability to build up a full-blown
multilib development environment (not covered here), but be aware that
going down this path can be a lot of trouble for your average DIY'er.
Everything else should remain as is.

If using the original (deprecated) build method, $DIY_ARCH is used to
determine the target architecture ($DIY_TARGET and $BIARCH are
ignored). Legal values for $DIY_ARCH are "i386", "ppc" and "x86_64"
depending on which architecture you are building on.

Ensure you are currently logged in as user `build' then become root
using plain `su'. DO NOT use `su
-' (ie: with the hyphen) because you'll lose the
environment variables that are critical for these commands to work as
intended.

Now exit the su (root) login and from this point onwards the environment
should be sane enough to build the temptools phase whenever you login as
user `build'. Dump all the needed tarballs and patches into their
respective dirs. Now you're ready to rock.

2.2. Tarballs

Here is a list of source packages used in the build. This information is
extracted into the aforementioned
packagedata file for easy
parsing and/or conversion into a wget script:

2.3. Patches

Patches used in the Reference Build can be downloaded from
http://downloads.diy-linux.org/patches/. Be sure to
look in the "PM" subdirectory for Package Management related patches,
"ppc" subdirectory for PowerPC related patches, etc.

3. Temptools Phase - Next Generation Build Method

Warning -- Work In Progress!!

This is the Next Generation build method which is now the default and
preferred option. The original method has been deprecated. The new
method of bootstrapping a Linux system works well for all supported
architectures, supports Package Management and has been ICA verified.
For now, only use recent toolchain combinations with this method i.e.
Glibc-2.6.x, GCC-4.2.x, Binutils-2.18 and newer. Please note - the
Rationales are not yet accurate as all the text was just copied from the
original method. This will be fixed progressively in due course.

3.1. Bash-3.2 Pass 1

Important

This first pass of Bash is the only package we install "by hand".
Everything else after this point is expected to be automated via a
build script. However, you can still drive the build manually by
typing in all the commands on the interactive shell command line if
you prefer. Note: Do not skip Bash
Pass 1. It used to be optional but is now a requirement of the build
method due to our use of CONFIG_SHELL (see below).

Installing Bash as the very first package allows us to specify the
"configure shell" to be used by all Autoconf-produced configure
scripts during the Temptools phase. This is achieved via the
CONFIG_SHELL environment variable mentioned earlier. It serves to
enhance the robustness of the build method by reducing reliance on
the host's /bin/sh thus avoiding potential
breakage
when bootstrapping from older distros.

Another reason for installing Bash first up is that we can write our
build script using all the features of modern Bash without having to
worry about which shell is installed as /bin/sh on the host. Simply
launch your build script like this: `bash-pass1 myscript.sh'.

Tip

There is additional benefit to using this technique in that it
allows greater flexibility for handling the transition between the
Temptools and Chroot phases. For example, you can set the
interpreter path in your build script to `#!/temptools/bin/bash'
which allows easy re-execution of the same script upon entering
the chroot environment without having to fuss over having a
`$SYSROOT/bin/sh' symlink already set up. Please refer to the
author's own
gsbuild
scripts for a sample implementation. Naturally, this point
is moot if you script in pure Bourne or handle the
Temptools/Chroot phases transition using a different method.

For our purposes of bootstrapping a new Linux system within the
Reference Build context, Bash should build fine on any sane distro
made within the last 5 years.

The "unset CONFIG_SHELL" in the configure invocation is needed for
configure to succeed because the shell pointed to by CONFIG_SHELL
doesn't exist yet i.e. we are installing it here.

Note: some older hosts have `libtermcap' installed and Bash will
prefer it over Ncurses if found by configure unless `--with-curses'
is given. Bash also comes with its own Termcap. Rather than tinker
with configure switches, it's actually more robust to just let
configure work it out for itself. This Pass 1 of Bash is simply for
running scripts during the Temptools phase so it shouldn't really
matter which Termcap is used.

Double ampersands are used here because we're still operating in an
interactive shell session at this point. We recommend that you
employ a proper error trapping strategy in your build script (eg:
set -e). Therefore, double ampersands are not provided in the build
commands from this point onwards.

3.2. Make-3.81 Pass 1

./configure
make
cp -v make ${TT_PFX}/bin
ln -sv make ${TT_PFX}/bin/gmake

Rationale Notes

for bootstrapping purposes, Make should build fine on any sane
distro made from 1999 onwards.

the Glibc build needs a version of `make' 3.79 or newer which older
distros don't have. We remove all doubts by installing the current
version. Why install it now rather than just prior to the Glibc
build? Well, it makes sense to integrate it from the outset so that
we can start using it immediately.

the `gmake' symlink is needed for robustness. `gmake' already exists
on some hosts which will prevent Glibc's configure script from
finding the `make' we install here.

3.3. Sed-4.1.5 Pass 1

for bootstrapping purposes, Sed should build fine on any sane distro
made from 1999 onwards.

The ac_cv_header_stdbool_h=yes tweak is needed when building on
older hosts. Please read
this
bug report for details. Note that on newer hosts the
variable is set automatically by configure thus making the tweak
unnecessary, but it's perfectly fine to leave it there in all
circumstances.

Why do we install a Pass 1 of Sed here? There have been reported
problems where the Glibc build chokes with older Sed versions. The
author hasn't personally seen any such problem but feels it's safer
to just install a current Sed from the outset. An added bonus is
that we can rely on using `sed -i' during the
entire Temptools phase thus simplifying scripting.

The `echo "MAKEINFO = :" >> Makefile'
tweak is used to remove dependence on `makeinfo' and therefore
eliminate all associated problems. Please read the mailing list
thread starting with
this
post for details.

We create a lib64 -> lib symlink in the x86_64 case because we
are not building a multi-arch toolchain. Even in single-arch mode
64-bit architectures tend to hardwire references to "lib64" in many
places throughout the toolchain. It's possible to force everything
to "lib" but we'd rather keep all the build commands compatible with
other architectures. The symlink keeps things sane and everything
Just Works™.

After the first Binutils are installed, -B/usr/bin/ is temporarily
added to the CC definition in order to force the host GCC to use the
host Binutils. Without this override, the host GCC will find the
newly installed Binutils in the PATH thus causing a toolchain
mismatch and potential breakage. This breakage is likely to strike
when using a bleeding edge distro as a host (eg: Fedora) and you're
building a toolchain comprised of older versions of toolchain
components than those on the host. The "if..echo..grep" statement is
required on those hosts with GCC-2.95.x or earlier installed. More
background on all of this can be found in the mailing list thread
starting with
this
post (continued
here).

Note: Unlike LFS, we don't statically link the Pass 1 of Binutils
because it's simply unnecessary, and in actual fact is a potential
source of failure. We therefore link dynamically for robustness
reasons. If you still think static linking is a good idea then at
least take note of the Glibc maintainer's
views
on the subject.

Because we're linking dynamically, there's no need to specify `make
configure-host' after the initial configure. However, when building
on SMP the top level Makefile will run the sub-configure scripts for
Libiberty and Binutils simultaneously. This is great for speed and
efficiency, but unfortunate for the build log which ends up all
jumbled. If you'd like the build log to remain neat and tidy (eg:
for diffing purposes), run the sub-configure scripts in serial by
executing make configure-host -j1 after the
initial configure.

GCC-4.3 and above requires the MPFR and GMP libraries. Here we are
taking advantage of the largely undocumented "combined tree" feature
of the GNU toolchain. Integrating MPFR and GMP into the GCC build
itself greatly simplifies matters and mostly removes all host issues
and other complications of building these libs separately. More
information in
this
mailing list post.

The switches `--disable-libmudflap', `--disable-libssp',
`--disable-libgomp' and `--disable-decimal-float' first appeared in
GCC-4.0, GCC-4.1, GCC-4.2 and GCC-4.3 respectively. They are used
here to streamline our initial "bootstrap" GCC. The switches are all
compatible within supported GCC versions ie: earlier versions simply
ignore the later switches.

Note: Starting with GCC-4.2, the top-level bootstrap mechanism
became the default. This means a plain "make" will now default to
performing a 3 stage bootstrap unless `--disable-bootstrap' is
given. The pre-GCC-4.2 method of "make bootstrap" behaves
identically. Therefore, we can maintain compatibility with earlier
versions by continuing to use "make bootstrap".

The libgcc_eh.a symlink is needed to satisfy the upcoming Glibc
build. Please read
this
mailing list post for some rationale discussion.

3.6. Linux-Kernel-Headers-2.6.27

Warning

The 2.6.18 kernel introduced the next generation method of obtaining
sanitized kernel headers. For some background on the issue please see
this mailing list
post. When using the new technique you
MUST SKIP the Linux-Libc-Headers step
which follows next. In other words, choose one or the other, not both.
Only use the old method when building a toolchain comprised of older
components (see table of toolchain combinations listed earlier).

3.9. Glibc (2.3.6 through 2.8)

Proceed with Caution!

Glibc is possibly the most critical piece of software you will compile
in a base Reference Build. There is a high probability of something
going wrong. Be sure to heed the advice contained in
Section B, “Glibc General Advice” before taking on this monster.
Note: the Glibc compiled here in the Temptools phase is merely a
"throw away bootstrap" Glibc, but please do read the aforementioned
advice anyway.

We need to fiddle with `-march=' in CFLAGS because as of Glibc-2.6,
i486 is the minimum supported configuration on x86. More information
here.

`--without-selinux' is a workaround needed on Fedora Core 3 hosts
(those with the "libselinux-devel" package installed) to prevent
build failure caused by a subtle Makefile bug. Upstream have been
notified
of the issue.

`AUTOCONF=no' is used here for robustness. It simply prevents
unnecessary Autoconf invocations when the Makefiles detect a
`configure' file older than its respective `configure.in'. This is
usually the case when working with CVS snapshots, but it also
applies to release tarballs prior to glibc-2.3.5 when the issue was
finally addressed. These unnecessary Autoconf invocations have been
known to cause build failures on Debian hosts. An alternative
approach is to run `find . -name configure | xargs touch' in the src
dir before running `configure'. Note: `--without-cvs' does not
address the issue.

`MAKEINFO=:' is used to prevent installation failures when the host
has an older `makeinfo'. Problems were first noticed with Glibc-2.7.

For x86_64 we need to empty the "multilib" spec to prevent GCC from
searching for libs on the host. The sed simply modifies the spec to
mirror how it would appear in a single-arch scenario. More
information in
this
mailing list post.

The tweak to STANDARD_STARTFILE_PREFIX_{1,2} ensures there is zero
chance of any libs and startfiles being found on the host. More
information in
this
mailing list post. Some older information related to the
same issue can be found
here.

As mentioned earlier, we need to pass `--disable-bootstrap' here to
prevent GCC-4.2 and above from performing a 3 stage bootstrap. The
switch is ignored by earlier versions and is therefore compatible.

The notes about `make configure-host' from Binutils Pass 1 and GCC
Pass 1 also apply here. However, it gets a little more complicated
because we're also building Libstdc++. To achieve the same outcome
(ie: run sub-configures in serial for neat build logs under SMP),
the sequence needs to be like this:

make configure-host -j1
make all-host
make configure-target -j1
make all-target

When building with GCC-4.x we use a sed to add
`-fomit-frame-pointer' to the GCC Makefile to address a correctness
issue. Please read
this
mailing list post for some rationale discussion.

3.14. Coreutils-6.12

For some background on the `--enable-install-program=hostname,su'
configure switch, please read
this
mailing list post.

Because we are operating as a non-privileged user, Coreutils will
suppress installation of the `su' program. This is signified by a
message shown during `make install': "WARNING: insufficient access;
not installing su NOTE: to install su, run 'make install-root' as
root". The reason for this is `su' needs to be installed setuid root
in order to be fully functional. We manually install it here because
even without the setuid bit, we can still use it when inside the
Chroot environment to (a) run various testsuites as a non-privileged
user and (b) employ Package Management techniques that require
building sources as a non-privileged user.

Important

From this point onwards if you need to become root you
must call the host's `su' explicitly
by running /bin/su. Just running
su will call the first `su' in the $PATH which
is the one we just installed (it will fail because of the missing
setuid bit). This is especially important when we go to enter the
Chroot environment at the beginning of the next phase.

3.28. M4-1.4.13

Tip

It's worth noting the next 3 packages (M4, Bison and Flex) are not
strictly necessary in the Temptools phase when using an official FSF
Binutils release. However, they are mandatory
if using a release from HJL. The reason is that FSF releases are made
with a "make dist" style procedure whereby the generated files (from
Bison and Flex) are already included in the tarball thus removing the
requirement for Bison and Flex at Binutils build time (in the Chroot
phase). Our recommendation is to always build M4, Bison and Flex here
to ensure the build recipe remains compatible with both FSF and HJL
Binutils releases. There is another issue: if you skip M4, Bison and
Flex here it's likely you'll run into
ICA
trouble which will require an additional
hack
when building Bison in the Chroot phase.

./configure
make V=1
cp -v src/m4 ${TT_PFX}/bin

3.29. Bison-2.4.1

./configure
make
make install

3.30. Flex-2.5.35

./configure
make
cp -v flex ${TT_PFX}/bin

3.31. E2fsprogs-1.41.4

mkdir build
cd build
../configure
make libs -j1
make install-libs

Rationale Notes

We install the E2fsprogs libraries here to satisfy the build of
`mount' from Util-Linux-ng in the next step. If you choose not to
build `mount' then these libraries are naturally not required.

The sed to `configure' forces Expect to call a plain `stty' instead
of `/bin/stty' or
even
worse, `/usr/local/bin/stty'. In this way the first `stty'
in the PATH will always be used. It also avoids the need to create a
`/bin/stty' symlink in Section 4.5, “Create Symlinks”.

3.36. DejaGnu-1.4.4

./configure
make install -j1

3.37. Fakeroot-1.11 (Optional)

As mentioned in the Introduction, if you're employing Package
Management by building and creating archives as a non-privileged
user, files in the resultant archives can sometimes end up with
wrong file ownerships and permissions. Fakeroot can provide a
solution to this problem. It acts as a wrapper around various
library functions and a typical invocation is like this:
fakeroot
commands-you-want-run-under-a-fake-root-environment.
Generally, you only want to be performing the commands associated
with installation under Fakeroot. Running commands associated with
configuring and building usually work, but the Fakeroot
documentation recommends against it. Never run the testsuites for
Glibc, Coreutils and Perl under Fakeroot because you will experience
failures. The Bash testsuite can also be problematic. Consider this
package optional if you're a) not employing Package Management b)
building as root or c) using a Package Manager that doesn't require
assistance from Fakeroot.

The sed is needed to force Fakeroot to load the correct Glibc when
inside the Chroot environment. Without it, Fakeroot will load the
Temptools Glibc.

Another package with similar functionality but having a slightly
different feature set is Pretendroot. Details on this package can be
found
here.
Note: we haven't tested Pretendroot in the context of a Reference
Build so your mileage may vary.

3.38. Zlib-1.2.3 (Optional)

Note -- Optional Package Management with Pacman

Only install the next 3 packages (Zlib, Libtar and Pacman) if you
intend implementing Pacman based package management.
Pacman is a simple
tar.gz-based package manager for Linux originally written for the
Arch Linux distro.
Please refer to the
gsbuild
scripts for a sample Pacman implementation. It's worth noting
the author personally uses Pacman which means it receives regular
testing within the Reference Build context.

3.41. Ogdlutils-20041124 (Optional)

Note -- Optional Package Management with BPM

Only install the next 3 packages (Ogdlutils, Cpio and BPM) if you
intend implementing BPM based package management. BPM is an extremely
compact package manager with the simplest spec file format imaginable.
It comes from the Bent
Linux distro. Please refer to the
gsbuild
scripts for a sample BPM implementation. BPM only receives
occasional testing within the Reference Build context and is included
here mainly as a proof of concept. (Note: the gsbuild BPM
implementation currently gets some file permissions wrong - FIXME).

make -C c libogdl gpath -j1
cp -v c/gpath ${TT_PFX}/bin

3.42. Cpio-2.9 (Optional)

./configure
make
cp -v src/cpio ${TT_PFX}/bin

3.43. BPM-1.7 (Optional)

cp -v bpm{build,install,pkgfix} ${TT_PFX}/bin

3.44. Remove Cruft

rm -rfv ${TT_PFX}/{,share/}{info,man}

4. Chroot Phase - Next Generation Build Method

Warning -- Work In Progress!!

This is the Next Generation build method which is now the default and
preferred option. The original method has been deprecated. The new
method of bootstrapping a Linux system works well for all supported
architectures, supports Package Management and has been ICA verified.
For now, only use recent toolchain combinations with this method i.e.
Glibc-2.6.x, GCC-4.2.x, Binutils-2.18 and newer. Please note - the
Rationales are not yet accurate as all the text was just copied from the
original method. This will be fixed progressively in due course.

4.1. About $PM_DEST

As mentioned in the Introduction, Package Management (PM) is a goal of
the Reference Build. Accordingly, all build commands in this Chroot
phase strive to be "PM friendly". This is reflected in those individual
commands associated with file installation. Commands associated with
configuring and building remain exactly the same as when performing a
"build in place" style installation.

You'll notice that a typical invocation of make
install has now become make DESTDIR=$PM_DEST
install. The expectation is this: if you're employing PM
then you'll assign a value to the PM_DEST variable in your scripting
environment that is equivalent to wherever your chosen Package Manager
expects to find the installed files. Doing it this way ensures the
installation commands remain generic. If you're not employing PM, that
is you're performing a usual "build in place" style installation, you
just have to ensure that the PM_DEST variable is nonexistent (or empty
or unset), then the installation commands will work exactly as if
`DESTDIR=$PM_DEST' weren't specified.

4.2. Enter Chroot Environment

Important

Earlier in the Temptools phase we installed a somewhat crippled `su'
program from Coreutils (with a missing setuid bit). Therefore, in
order to become root and enter the Chroot environment, you
must call the host's `su' explicitly
by running /bin/su or else you'll fail to gain
root privilege.

Before entering the chroot environment, be sure to become root using
plain `/bin/su'. DO NOT use `/bin/su
-' (ie: with the hyphen) because you'll lose the
environment variables that are critical for the following chroot
command to work as intended. When including this chroot command in
your script you'll need to make 2 adjustments: (a) you should omit the
$PS1 variable because it is normally unset in non-interactive shells
and (b) you'll want to execute a script instead of
${TT_PFX}/bin/bash --login +h (make sure your
script does a set +h to switch off hashing).

There really shouldn't be any need to create a whole directory
layout from the outset. Just create the bare minimum to get the
build going then let the packages themselves create the rest. If you
want a full FHS/LSB compliant layout then just add the missing bits
after the build. Consider this experimental. FIXME

Current LFS inefficiently uses a raft of `install -d' commands for
directory creation here. There is no need for this when `mkdir' is
perfectly suited to the task. Only when custom permissions are
required do we use `install -d'.

The command to create compatibility symlinks in /usr is commented
out above. We force packages to install man and info files into
proper FHS compliant locations by globally setting defaults in
config.site (see below). Those few packages that don't pick up the
defaults are taken care of individually in the build commands. Feel
free to uncomment the above line if you still prefer to keep the
compatibility symlinks.

The `/bin/echo' symlink is required for the Glibc-2.6 (and above)
testsuite to pass. More information
here.

The `libstdc++.so.6' symlink is needed to satisfy the Glibc-2.4 (and
above) testsuite. More information
here.

The `libfakeroot.so.0' symlink is only needed if you installed the
optional Fakeroot package. If you build Glibc under Fakeroot (not
recommended) the build will fail in the absence of this symlink.
Additionally, when Fakeroot is used to wrap file
installation commands only (its preferred mode of
operation), the symlink prevents some ugly non-fatal errors during
Glibc and GCC installation eg: "ERROR: ld.so: object
'libfakeroot.so.0' from LD_PRELOAD cannot be preloaded: ignored."

If you're employing Package Management, the chances are you'll
probably want to build packages as a non-privileged user. Here we
setup a user called "pkgmgr" for exactly this purpose. You can use
the `su' program (installed earlier) to build packages as the
Package Management User (PMU). To build a package as the PMU,
something like: su pkgmgr -c
"command-to-build-package" should achieve the desired
effect. Creating the PMU is of course optional if you'd prefer to
build the Chroot phase as root.

We also create a dummy user and some dummy groups for the benefit of
those testsuites that need to be run as a non-privileged user. These
entries should be removed upon completion of the build.

4.8. Makedev-1.7

We install MAKEDEV here primarily for the purpose of
bootstrapping the Reference Build. That is, you don't
have to deploy the MAKEDEV script and the "on disk" device special
files it created in your final file system image. Some folks prefer
to use the (now deprecated) kernel based Devfs facility to manage
/dev. However, lately the general trend has moved towards the Udev
system which runs entirely in userspace. Other options for
populating /dev during the chroot phase of the Reference Build
include: (a) performing a `mount --bind' of the host's /dev onto
$SYSROOT/dev (b) manually mknod'ing the bare minimum device special
files needed to satisfy the build or (c) running the `udevstart'
program from Udev. All of these solutions are suboptimal in the
author's opinion which is why using MAKEDEV here is recommended.
What you do to manage /dev in the final deployed system is entirely
your choice, but if you're after the latest and greatest, Udev would
be your best bet.

4.9. Man-Pages-3.20

rm -fv man3/getspnam.3 man5/passwd.5make DESTDIR=$PM_DEST install

Rationale Notes

In general terms, GNU prefers the Info documentation system over man
pages. However, many GNU packages install both info pages and (in
some cases, less than adequate) man pages. The man pages listed
above will be installed later on by Coreutils, Diffutils and Shadow
and are therefore superfluous. We suppress their installation here
because some Package Managers will refuse to install an archive if a
file conflict is detected. If you'd prefer to keep the copies here,
omit the rm statements and somehow suppress the installation of said
man pages when installing the 3 aforementioned packages later on.

4.10. Man-Pages-Posix-2003-a

make DESTDIR=$PM_DEST install

4.11. Linux-Kernel-Headers-2.6.27

Warning

As mentioned earlier, the 2.6.18 kernel introduced the next generation
method of obtaining sanitized kernel headers. When using the new
technique you MUST SKIP the Linux-Libc-Headers
step which follows next.

4.14. Glibc (2.3.6 through 2.8)

Proceed with Caution!

Glibc is possibly the most critical piece of software you will compile
in a base Reference Build. There is a high probability of something
going wrong. Be sure to heed the advice contained in
Section B, “Glibc General Advice” before taking on this monster.

For an explanation of why we add `-mtune=generic' to CFLAGS in the
x86 GCC-4.2 and above case, please read
this
mailing list post.

Whenever Glibc is installed directly into a prefix of /usr via `make
install' (ie: the Makefile variable `install_root' is unset), a
script called `test-installation.pl' is run to perform some basic
sanity checks (compiling, linking, executing, etc) against the newly
installed Glibc. If all goes well, a message "Your new glibc
installation seems to be ok." will be shown. However, at the time
the script is run our toolchain defaults are still pointing to
$TT_PFX. In particular, ld's library search path is pointing to
$TT_PFX/lib and gcc's dynamic linker spec is pointing to
$TT_PFX${DL}. In actual fact, this means the checks are made against
the wrong Glibc. The problem will usually go unnoticed because the
Glibc's in $TT_PFX/lib and /usr/lib are functionally equivalent and
the checks always pass. You can expose the problem by building
different Glibc addons in each phase eg: configure the Temptools
phase Glibc with `--enable-add-ons=nptl' and configure the Chroot
phase Glibc with `--enable-add-ons=nptl,libidn' (assuming you have
downloaded and unpacked the separate addon for Libidn). We work
around the issue by using a sed to insert `-L/usr/lib
-Wl,-dynamic-linker=${DL}' at an appropriate place. Note that ld's
library search path and the gcc dynamic linker spec will be
readjusted to point to the new Glibc in the next step.

The `gconv-modules' modules tweak is a workaround for the testsuite.
Backgroud information can be found
here.

The command to install the localedata has been commented out above.
If you need all the locales, unlikely as that may be, feel free to
uncomment the relevant line. It used to be the case that some of
these locales were required for the Libstdc++ testsuite to pass. But
that is no longer true with the advent of GCC-3.4. More information
about this issue is
here.
However, the Libstdc++ docs do recommend a minimum list of locales,
so having them installed should result in wider Libstdc++ testsuite
coverage. Note that you can install individual locales at any time
using the `localedef' command ie: locale installation doesn't have
to occur as part of the Glibc install. At the very least, you'll
probably want to install a locale for your own country.

The warning in current LFS about unsetting CFLAGS and not overriding
the default optimization for Glibc is a crock. Glibc is EXACTLY the
kind of package that should be optimized - Refer Golden Rule No. 2
in Section B, “Glibc General Advice”. The warning in LFS dates back
to times when clues were scarce and tools weren't as good.

Tip -- Set up Dual Threading Libraries

Ever since the advent of NPTL most mainstream distros are set up with
both NPTL and Linuxthreads enabled Glibc's installed side by side.
This is for backwards compatibility reasons. NPTL is still relatively
new and a lot of commercial software simply won't run under it. Be
sure to check out Section B, “Dual Threading Libraries” if you need
this backwards compatibility and want to set up your system to match
the distros.

We build Zlib here in the build order to satisfy Binutils starting
with version 2.19. More rationale in
this
mailing list post.

The sed to `configure' replicates a fix from upstream which ensures
the shared lib is always built with -fPIC. Currently, -fPIC is lost
if CFLAGS happen to be set in the environment. More information in
this
mailing list post. Note: Zlib beta versions can be found
here.

The reason for the mv -v {,x}libz.a etc. is
to allow better locality of the "installation" commands. More
rationale in
this
mailing list post.

A newer `standards.info' is installed later by the Autoconf package.
Here we use `rm' and a Makefile sed to suppress installation of the
unwanted file. The `rm' is needed when using FSF releases but is
otherwise harmless when using HJL releases.

4.18. GCC (3.4.6 through 4.3.3)

Caution -- Check your GCC Testsuite Results!

Upon completion of the GCC Testsuite run, sanity check your results by
comparing against a set of known good results as per the links in the
following table:

Note: if you're using GCC-3.4.4 or earlier and running under a 2.6.12
kernel or above, a new kernel feature known as "address space
randomization" will likely cause some PCH tests to fail as per these
unsatisfactory
results. The problem is now resolved in GCC-3.4.5 and above.
However, if for some reason you need to use an old GCC version, the
problem can be worked around by following the advice contained in
this
mailing list post.

When building with GCC-4.x we use a sed to add
`-fomit-frame-pointer' to the GCC Makefile to address a correctness
issue. Please read
this
mailing list post for some rationale discussion.

When building with GCC-4.x we use a sed to increase the timeouts on
2 libmudflap tests to avoid spurious failures. The problem is most
noticeable when running the GCC testsuite in parallel on SMP.
Upstream have been
notified of the
issue. Note the timeouts were increased from 3 to 10 seconds as of
GCC-4.1 but this is still not enough. The sed is compatible across
all supported GCC versions.

We add "MAKEINFO=makeinfo" and "gcc_cv_prog_makeinfo_modern=yes" to
the `make' invocation to ensure installation of the GCC info pages.
Please read
this
mailing list post for background details.

For an explanation of the Makefile.in sed, configure switches and
chown command, please read
this
mailing list post.

The sed to `tests/other-fs-tmpdir' adds /dev/shm as a location to
search when the testsuite goes looking for a writable directory on a
different disk partition. This allows some tests to run that would
otherwise be skipped. Passing `CANDIDATE_TMP_DIRS=/dev/shm' would
achieve the same effect but the sed saves us passing it twice.

A bunch of commands are moved to /bin as mandated by the FHS. In the
case of `[' and `test', the FHS says they must be together in either
/bin or /usr/bin. We leave them in /usr/bin to match the big
distros. Even though `sleep' and `touch' are not mentioned by the
FHS, we move them to /bin because: (a) they are commonly used in
bootscripts, (b) they have always traditionally been there and (c)
this matches the big distros. Please read
this
mailing list post for some rationale discussion.

We have opted to use the `su' from Coreutils rather than the one
from Shadow. Please read
this
mailing list post for some rationale discussion. If you
would rather use the `su' from Shadow, simply omit `su' from the
--enable-install-program configure switch and the 4th mv command.

If you would like `uname' to print something useful for the `-p' and
`-i' options instead of "unknown", 3rd party patches are known to
exist (or at least have existed in the past) at the
LFS
and
Gentoo
projects. But before applying any such hacks, ask yourself this
question: Why haven't these types of patches been integrated by the
upstream Coreutils developers? Some insight into this issue can be
found in
this
mailing list thread.

4.22. Iana-Etc-2.30

make STRIP=yes
make testmake DESTDIR=$PM_DEST install

Rationale Notes

Note: You should be aware there are performance implications when
using this package because it results in an enormous /etc/services
file. Please read
this
mailing list post for some background discussion.

4.33. Gettext-0.17

The Gettext testsuite takes a painfully long time to run, and seeing
as Gettext is not a critical component in the overall scheme of
things, running this testsuite is considered optional. You can
reduce the pain somewhat by only running the less time consuming
parts of the testsuite like so: make -C gettext-tools
check.

Note that the Gettext testsuite will gain wider coverage if you have
a bunch of various locales installed (the testsuite will inform you
of which ones are missing).

Note also that 2 tests will fail if Gettext is configured with
`--disable-nls' and the fr_FR locale is installed. More information
about this issue is
here.

4.42. Libtool-2.2.6a

4.43. Autoconf-2.63

4.44. Automake-1.10.1

./configure
make
make -k check || :make DESTDIR=$PM_DEST install

This release of Automake contains an expected testsuite failure when
run as root. More information
here.

4.45. Bash-3.2

Important

The Bash testsuite is unlike most others in that it doesn't stop if
errors are encountered. This means you have to physically examine the
output and cannot rely on the exit status of the `make tests' command.
It even tells you as much at the beginning of the run: "Any output
from any test, unless otherwise noted, indicates a possible anomaly".

Testing kernel module handling in a live kernel cannot be done
easily for obvious reasons. Therefore the Module-Init-Tools
developers have devised a custom testsuite procedure that involves
separately compiling the package with a set of stubs (see
testing.h). The net result of this means we must first run the
testsuite, perform a `make distclean' then compile the package for
real.

Please read the mailing list thread starting with
this
post for some rationale behind the above build commands.

The `bin_PROGRAMS=login suidbins=' addition to the `make install'
invocation is used to suppress installation of `groups' and `su'.
This is because we have opted to use the versions from Coreutils
instead. In the case of `su', please read
this
mailing list post for some rationale discussion. If you
would rather use the `su' from Shadow, replace the installation line
above with make DESTDIR=$PM_DEST bin_PROGRAMS="login su"
install. You would also want to adjust the sed to
`man/Makefile' to ensure installation of the su related man pages.

The sed to `stage2/size_test' is used to work around an issue with a
failing ufs2 test. Just ignoring the failure is unsatisfactory
because the `size_test' script bails out on the first failure and
therefore doesn't test the remaining file systems (fat, e2fs, minix)
2 of which are clearly important.

The "256 byte inode" patch could be considered optional in some
circumstances. But it's probably best to apply it. More detailed
rationale discussion can be found in
this
mailing list thread.

The Grub build is sensitive to non-standard CFLAGS in the
environment. Seeing as we have already set our CFLAGS globally in
`config.site', we force Grub to use its own defaults by configuring
with `default_CFLAGS=yes'.

4.68. Cpio-2.9 (Optional)

4.69. BPM-1.7 (Optional)

make DESTDIR=$PM_DEST install

5. Finishing Up

5.1. Making it Functional

FIXME

Mention some brief stuff here about kernel compilation and installation,
bootscripts, network setup, system config files, fstab, bootloader, etc,
etc, etc. In other words, anything needed to make the built system
functional.

Remove the dummy user and groups that were created earlier.

sed -i '/dummy/d' /etc/{group,passwd}

For enhanced security, you probably want to enable shadow passwords and
shadow group passwords:

/usr/sbin/pwconv
/usr/sbin/grpconv

5.2. Conclusion

If you found this document useful, I'd appreciate you dropping me a line
at <gschafer@zip.com.au>. It will help me gauge levels of
interest in the project and determine future development direction.

5.3. Acknowledgments

Many folks have helped out with fixes, suggestions, bug reports and
testing (too numerous to list here). But some have made outstanding
contributions for which I'd like to especially say thanks:

6. Temptools Phase - Original Build Method (Deprecated)

Warning -- Original Build Method Now Deprecated

What follows is the original build method which is now deprecated. It no
longer receives any testing from the author and it only receives
mechanical updates which renders it essentially unmaintained. In
particular, the original method will not be updated for GCC 4.3. Please
note that it will be removed entirely in due course. The Next Generation
build method is now the preferred option.

6.1. Bash-3.2 Pass 1

Important

This first pass of Bash is the only package we install "by hand".
Everything else after this point is expected to be automated via a
build script. However, you can still drive the build manually by
typing in all the commands on the interactive shell command line if
you prefer. Note: Do not skip Bash
Pass 1. It used to be optional but is now a requirement of the build
method due to our use of CONFIG_SHELL (see below).

Installing Bash as the very first package allows us to specify the
"configure shell" to be used by all Autoconf-produced configure
scripts during the Temptools phase. This is achieved via the
CONFIG_SHELL environment variable mentioned earlier. It serves to
enhance the robustness of the build method by reducing reliance on
the host's /bin/sh thus avoiding potential
breakage
when bootstrapping from older distros.

Another reason for installing Bash first up is that we can write our
build script using all the features of modern Bash without having to
worry about which shell is installed as /bin/sh on the host. Simply
launch your build script like this: `bash-pass1 myscript.sh'.

Tip

There is additional benefit to using this technique in that it
allows greater flexibility for handling the transition between the
Temptools and Chroot phases. For example, you can set the
interpreter path in your build script to `#!/temptools/bin/bash'
which allows easy re-execution of the same script upon entering
the chroot environment without having to fuss over having a
`$SYSROOT/bin/sh' symlink already set up. Please refer to the
author's own
gsbuild
scripts for a sample implementation. Naturally, this point
is moot if you script in pure Bourne or handle the
Temptools/Chroot phases transition using a different method.

For our purposes of bootstrapping a new Linux system within the
Reference Build context, Bash should build fine on any sane distro
made within the last 5 years.

The "unset CONFIG_SHELL" in the configure invocation is needed for
configure to succeed because the shell pointed to by CONFIG_SHELL
doesn't exist yet i.e. we are installing it here.

Note: some older hosts have `libtermcap' installed and Bash will
prefer it over Ncurses if found by configure unless `--with-curses'
is given. Bash also comes with its own Termcap. Rather than tinker
with configure switches, it's actually more robust to just let
configure work it out for itself. This Pass 1 of Bash is simply for
running scripts during the Temptools phase so it shouldn't really
matter which Termcap is used.

Double ampersands are used here because we're still operating in an
interactive shell session at this point. We recommend that you
employ a proper error trapping strategy in your build script (eg:
set -e). Therefore, double ampersands are not provided in the build
commands from this point onwards.

6.2. Make-3.81 Pass 1

./configure
make
cp -v make ${TT_PFX}/bin
ln -sv make ${TT_PFX}/bin/gmake

Rationale Notes

for bootstrapping purposes, Make should build fine on any sane
distro made from 1999 onwards.

the Glibc build needs a version of `make' 3.79 or newer which older
distros don't have. We remove all doubts by installing the current
version. Why install it now rather than just prior to the Glibc
build? Well, it makes sense to integrate it from the outset so that
we can start using it immediately.

the `gmake' symlink is needed for robustness. `gmake' already exists
on some hosts which will prevent Glibc's configure script from
finding the `make' we install here.

6.3. Sed-4.1.5 Pass 1

for bootstrapping purposes, Sed should build fine on any sane distro
made from 1999 onwards.

The ac_cv_header_stdbool_h=yes tweak is needed when building on
older hosts. Please read
this
bug report for details. Note that on newer hosts the
variable is set automatically by configure thus making the tweak
unnecessary, but it's perfectly fine to leave it there in all
circumstances.

Why do we install a Pass 1 of Sed here? There have been reported
problems where the Glibc build chokes with older Sed versions. The
author hasn't personally seen any such problem but feels it's safer
to just install a current Sed from the outset. An added bonus is
that we can rely on using `sed -i' during the
entire Temptools phase thus simplifying scripting.

The `echo "MAKEINFO = :" >> Makefile'
tweak is used to remove dependence on `makeinfo' and therefore
eliminate all associated problems. Please read the mailing list
thread starting with
this
post for details.

We create a lib64 -> lib symlink in the x86_64 case because we
are not building a multi-arch toolchain. Even in single-arch mode
64-bit architectures tend to hardwire references to "lib64" in many
places throughout the toolchain. It's possible to force everything
to "lib" but we'd rather keep all the build commands compatible with
other architectures. The symlink keeps things sane and everything
Just Works™.

After the first Binutils are installed, -B/usr/bin/ is temporarily
added to the CC definition in order to force the host GCC to use the
host Binutils. Without this override, the host GCC will find the
newly installed Binutils in the PATH thus causing a toolchain
mismatch and potential breakage. This breakage is likely to strike
when using a bleeding edge distro as a host (eg: Fedora) and you're
building a toolchain comprised of older versions of toolchain
components than those on the host. The "if..echo..grep" statement is
required on those hosts with GCC-2.95.x or earlier installed. More
background on all of this can be found in the mailing list thread
starting with
this
post (continued
here).

Note: Unlike LFS, we don't statically link the Pass 1 of Binutils
because it's simply unnecessary, and in actual fact is a potential
source of failure. We therefore link dynamically for robustness
reasons. If you still think static linking is a good idea then at
least take note of the Glibc maintainer's
views
on the subject.

Because we're linking dynamically, there's no need to specify `make
configure-host' after the initial configure. However, when building
on SMP the top level Makefile will run the sub-configure scripts for
Libiberty and Binutils simultaneously. This is great for speed and
efficiency, but unfortunate for the build log which ends up all
jumbled. If you'd like the build log to remain neat and tidy (eg:
for diffing purposes), run the sub-configure scripts in serial by
executing make configure-host -j1 after the
initial configure.

We apply a sed to remove STAGE1_CHECKING from the GCC Makefile and
also pass `--disable-checking' to configure in order to considerably
reduce bootstrap time. Please read
this
mailing list post for some rationale discussion. Note: 4.2.X
introduced `--disable-stage1-checking' which achieves the same
effect as the sed. The switch is compatible within supported GCC
versions ie: earlier versions simply ignore this switch. The sed
doesn't work on 4.2.X but is harmless nevertheless. More info in
this
mailing list post.

Just as in Binutils Pass 1, we again need the CC="gcc -B/usr/bin/"
tweak for the same reasons mentioned there. Our new GCC will start
combining with our new Binutils in stage 2 of the 3 stage GCC
bootstrap procedure.

No static linking here for the same reasons as mentioned above in
Binutils Pass 1. Additionally, the note about `make configure-host'
from there also applies here (Binutils and GCC share the same top
level build machinery).

When building with GCC-4.x the Makefile variable BOOT_CFLAGS
contains `-O2 -g -fomit-frame-pointer' by default. Because we are
passing BOOT_CFLAGS to `make' by hand (basically to avoid building
with debugging information) we must add `-fomit-frame-pointer' to
BOOT_CFLAGS to ensure the compiler is built as intended by the GCC
developers. Please read
this
mailing list post for some background discussion.

The switches `--disable-libmudflap', `--disable-libssp' and
`--disable-libgomp' first appeared in GCC-4.0, GCC-4.1 and GCC-4.2
respectively. They are used here to streamline our initial
"bootstrap" GCC. The switches are all compatible within supported
GCC versions ie: earlier versions simply ignore the later switches.

We pass `--disable-multilib' here (and in subsequent GCC builds)
because it's required to achieve a pure 64-bit build on x86_64. It
also speeds up the PowerPC build by omitting the "nof" directory
(ie: applicable to -msoft-float). The switch has no effect on x86
and is therefore compatible.

Note: Starting with GCC-4.2, the top-level bootstrap mechanism
became the default. This means a plain "make" will now default to
performing a 3 stage bootstrap unless `--disable-bootstrap' is
given. The pre-GCC-4.2 method of "make bootstrap" behaves
identically. Therefore, we can maintain compatibility with earlier
versions by continuing to use "make bootstrap".

The libgcc_eh.a symlink is needed to satisfy the upcoming Glibc
build. Please read
this
mailing list post for some rationale discussion.

6.6. Linux-Kernel-Headers-2.6.27

Warning

The 2.6.18 kernel introduced the next generation method of obtaining
sanitized kernel headers. For some background on the issue please see
this mailing list
post. When using the new technique you
MUST SKIP the Linux-Libc-Headers step
which follows next. In other words, choose one or the other, not both.
Only use the old method when building a toolchain comprised of older
components (see table of toolchain combinations listed earlier).

6.7. Linux-Libc-Headers-2.6.12.0 (Older toolchains only)

6.8. Glibc (2.3.6 through 2.7)

Proceed with Caution!

Glibc is possibly the most critical piece of software you will compile
in a base Reference Build. There is a high probability of something
going wrong. Be sure to heed the advice contained in
Section B, “Glibc General Advice” before taking on this monster.
Note: the Glibc compiled here in the Temptools phase is merely a
"throw away bootstrap" Glibc, but please do read the aforementioned
advice anyway.

We need to fiddle with `-march=' in CFLAGS because as of Glibc-2.6,
i486 is the minimum supported configuration on x86. More information
here.

`--without-selinux' is a workaround needed on Fedora Core 3 hosts
(those with the "libselinux-devel" package installed) to prevent
build failure caused by a subtle Makefile bug. Upstream have been
notified
of the issue.

`AUTOCONF=no' is used here for robustness. It simply prevents
unnecessary Autoconf invocations when the Makefiles detect a
`configure' file older than its respective `configure.in'. This is
usually the case when working with CVS snapshots, but it also
applies to release tarballs prior to glibc-2.3.5 when the issue was
finally addressed. These unnecessary Autoconf invocations have been
known to cause build failures on Debian hosts. An alternative
approach is to run `find . -name configure | xargs touch' in the src
dir before running `configure'. Note: `--without-cvs' does not
address the issue.

`MAKEINFO=:' is used to prevent installation failures when the host
has an older `makeinfo'. Problems were first noticed with Glibc-2.7.

For x86_64 we need to empty the "multilib" spec to prevent GCC from
searching for libs on the host. The sed simply modifies the spec to
mirror how it would appear in a single-arch scenario. More
information in
this
mailing list post.

On x86_64 we sed out the MULTILIB_OSDIRNAMES reference which has the
same effect as emptying the "multilib" spec. This is done for the
same reason mentioned earlier i.e. prevent GCC from searching for
libs on the host. More information in
this
mailing list post.

As mentioned earlier, we need to pass `--disable-bootstrap' here to
prevent GCC-4.2 and above from performing a 3 stage bootstrap. The
switch is ignored by earlier versions and is therefore compatible.

The notes about `make configure-host' from Binutils Pass 1 and GCC
Pass 1 also apply here. However, it gets a little more complicated
because we're also building Libstdc++. To achieve the same outcome
(ie: run sub-configures in serial for neat build logs under SMP),
the sequence needs to be like this:

make configure-host -j1
make all-host
make configure-target -j1
make all-target

When building with GCC-4.x we use a sed to add
`-fomit-frame-pointer' to the GCC Makefile to address a correctness
issue. Please read
this
mailing list post for some rationale discussion.

6.13. Coreutils-6.12

For some background on the `--enable-install-program=hostname,su'
configure switch, please read
this
mailing list post.

Because we are operating as a non-privileged user, Coreutils will
suppress installation of the `su' program. This is signified by a
message shown during `make install': "WARNING: insufficient access;
not installing su NOTE: to install su, run 'make install-root' as
root". The reason for this is `su' needs to be installed setuid root
in order to be fully functional. We manually install it here because
even without the setuid bit, we can still use it when inside the
Chroot environment to (a) run various testsuites as a non-privileged
user and (b) employ Package Management techniques that require
building sources as a non-privileged user.

Important

From this point onwards if you need to become root you
must call the host's `su' explicitly
by running /bin/su. Just running
su will call the first `su' in the $PATH which
is the one we just installed (it will fail because of the missing
setuid bit). This is especially important when we go to enter the
Chroot environment at the beginning of the next phase.

6.27. M4-1.4.13

Tip

It's worth noting the next 3 packages (M4, Bison and Flex) are not
strictly necessary in the Temptools phase when using an official FSF
Binutils release. However, they are mandatory
if using a release from HJL. The reason is that FSF releases are made
with a "make dist" style procedure whereby the generated files (from
Bison and Flex) are already included in the tarball thus removing the
requirement for Bison and Flex at Binutils build time (in the Chroot
phase). Our recommendation is to always build M4, Bison and Flex here
to ensure the build recipe remains compatible with both FSF and HJL
Binutils releases. There is another issue: if you skip M4, Bison and
Flex here it's likely you'll run into
ICA
trouble which will require an additional
hack
when building Bison in the Chroot phase.

./configure
make V=1
cp -v src/m4 ${TT_PFX}/bin

6.28. Bison-2.4.1

./configure
make
make install

6.29. Flex-2.5.35

./configure
make
cp -v flex ${TT_PFX}/bin

6.30. E2fsprogs-1.41.4

mkdir build
cd build
../configure
make libs -j1
make install-libs

Rationale Notes

We install the E2fsprogs libraries here to satisfy the build of
`mount' from Util-Linux-ng in the next step. If you choose not to
build `mount' then these libraries are naturally not required.

The sed to `configure' forces Expect to call a plain `stty' instead
of `/bin/stty' or
even
worse, `/usr/local/bin/stty'. In this way the first `stty'
in the PATH will always be used. It also avoids the need to create a
`/bin/stty' symlink in Section 7.5, “Create Symlinks”.

6.35. DejaGnu-1.4.4

./configure
make install -j1

6.36. Fakeroot-1.11 (Optional)

As mentioned in the Introduction, if you're employing Package
Management by building and creating archives as a non-privileged
user, files in the resultant archives can sometimes end up with
wrong file ownerships and permissions. Fakeroot can provide a
solution to this problem. It acts as a wrapper around various
library functions and a typical invocation is like this:
fakeroot
commands-you-want-run-under-a-fake-root-environment.
Generally, you only want to be performing the commands associated
with installation under Fakeroot. Running commands associated with
configuring and building usually work, but the Fakeroot
documentation recommends against it. Never run the testsuites for
Glibc, Coreutils and Perl under Fakeroot because you will experience
failures. The Bash testsuite can also be problematic. Consider this
package optional if you're a) not employing Package Management b)
building as root or c) using a Package Manager that doesn't require
assistance from Fakeroot.

The sed is needed to force Fakeroot to load the correct Glibc when
inside the Chroot environment. Without it, Fakeroot will load the
Temptools Glibc.

Another package with similar functionality but having a slightly
different feature set is Pretendroot. Details on this package can be
found
here.
Note: we haven't tested Pretendroot in the context of a Reference
Build so your mileage may vary.

6.37. Zlib-1.2.3 (Optional)

Note -- Optional Package Management with Pacman

Only install the next 3 packages (Zlib, Libtar and Pacman) if you
intend implementing Pacman based package management.
Pacman is a simple
tar.gz-based package manager for Linux originally written for the
Arch Linux distro.
Please refer to the
gsbuild
scripts for a sample Pacman implementation. It's worth noting
the author personally uses Pacman which means it receives regular
testing within the Reference Build context.

6.40. Ogdlutils-20041124 (Optional)

Note -- Optional Package Management with BPM

Only install the next 3 packages (Ogdlutils, Cpio and BPM) if you
intend implementing BPM based package management. BPM is an extremely
compact package manager with the simplest spec file format imaginable.
It comes from the Bent
Linux distro. Please refer to the
gsbuild
scripts for a sample BPM implementation. BPM only receives
occasional testing within the Reference Build context and is included
here mainly as a proof of concept. (Note: the gsbuild BPM
implementation currently gets some file permissions wrong - FIXME).

make -C c libogdl gpath -j1
cp -v c/gpath ${TT_PFX}/bin

6.41. Cpio-2.9 (Optional)

./configure
make
cp -v src/cpio ${TT_PFX}/bin

6.42. BPM-1.7 (Optional)

cp -v bpm{build,install,pkgfix} ${TT_PFX}/bin

6.43. Remove Cruft

rm -rfv ${TT_PFX}/{,share/}{info,man}

7. Chroot Phase - Original Build Method (Deprecated)

Warning -- Original Build Method Now Deprecated

What follows is the original build method which is now deprecated. It no
longer receives any testing from the author and it only receives
mechanical updates which renders it essentially unmaintained. In
particular, the original method will not be updated for GCC 4.3. Please
note that it will be removed entirely in due course. The Next Generation
build method is now the preferred option.

7.1. About $PM_DEST

As mentioned in the Introduction, Package Management (PM) is a goal of
the Reference Build. Accordingly, all build commands in this Chroot
phase strive to be "PM friendly". This is reflected in those individual
commands associated with file installation. Commands associated with
configuring and building remain exactly the same as when performing a
"build in place" style installation.

You'll notice that a typical invocation of make
install has now become make DESTDIR=$PM_DEST
install. The expectation is this: if you're employing PM
then you'll assign a value to the PM_DEST variable in your scripting
environment that is equivalent to wherever your chosen Package Manager
expects to find the installed files. Doing it this way ensures the
installation commands remain generic. If you're not employing PM, that
is you're performing a usual "build in place" style installation, you
just have to ensure that the PM_DEST variable is nonexistent (or empty
or unset), then the installation commands will work exactly as if
`DESTDIR=$PM_DEST' weren't specified.

7.2. Enter Chroot Environment

Important

Earlier in the Temptools phase we installed a somewhat crippled `su'
program from Coreutils (with a missing setuid bit). Therefore, in
order to become root and enter the Chroot environment, you
must call the host's `su' explicitly
by running /bin/su or else you'll fail to gain
root privilege.

Before entering the chroot environment, be sure to become root using
plain `/bin/su'. DO NOT use `/bin/su
-' (ie: with the hyphen) because you'll lose the
environment variables that are critical for the following chroot
command to work as intended. When including this chroot command in
your script you'll need to make 2 adjustments: (a) you should omit the
$PS1 variable because it is normally unset in non-interactive shells
and (b) you'll want to execute a script instead of
${TT_PFX}/bin/bash --login +h (make sure your
script does a set +h to switch off hashing).

There really shouldn't be any need to create a whole directory
layout from the outset. Just create the bare minimum to get the
build going then let the packages themselves create the rest. If you
want a full FHS/LSB compliant layout then just add the missing bits
after the build. Consider this experimental. FIXME

Current LFS inefficiently uses a raft of `install -d' commands for
directory creation here. There is no need for this when `mkdir' is
perfectly suited to the task. Only when custom permissions are
required do we use `install -d'.

The command to create compatibility symlinks in /usr is commented
out above. We force packages to install man and info files into
proper FHS compliant locations by globally setting defaults in
config.site (see below). Those few packages that don't pick up the
defaults are taken care of individually in the build commands. Feel
free to uncomment the above line if you still prefer to keep the
compatibility symlinks.

7.5. Create Symlinks

The `/bin/echo' symlink is required for the Glibc-2.6 (and above)
testsuite to pass. More information
here.

The `libstdc++.so.6' symlink is needed to satisfy the Glibc-2.4 (and
above) testsuite. More information
here.

The `libfakeroot.so.0' symlink is only needed if you installed the
optional Fakeroot package. If you build Glibc under Fakeroot (not
recommended) the build will fail in the absence of this symlink.
Additionally, when Fakeroot is used to wrap file
installation commands only (its preferred mode of
operation), the symlink prevents some ugly non-fatal errors during
Glibc and GCC installation eg: "ERROR: ld.so: object
'libfakeroot.so.0' from LD_PRELOAD cannot be preloaded: ignored."

If you're employing Package Management, the chances are you'll
probably want to build packages as a non-privileged user. Here we
setup a user called "pkgmgr" for exactly this purpose. You can use
the `su' program (installed earlier) to build packages as the
Package Management User (PMU). To build a package as the PMU,
something like: su pkgmgr -c
"command-to-build-package" should achieve the desired
effect. Creating the PMU is of course optional if you'd prefer to
build the Chroot phase as root.

We also create a dummy user and some dummy groups for the benefit of
those testsuites that need to be run as a non-privileged user. These
entries should be removed upon completion of the build.

7.8. Makedev-1.7

We install MAKEDEV here primarily for the purpose of
bootstrapping the Reference Build. That is, you don't
have to deploy the MAKEDEV script and the "on disk" device special
files it created in your final file system image. Some folks prefer
to use the (now deprecated) kernel based Devfs facility to manage
/dev. However, lately the general trend has moved towards the Udev
system which runs entirely in userspace. Other options for
populating /dev during the chroot phase of the Reference Build
include: (a) performing a `mount --bind' of the host's /dev onto
$SYSROOT/dev (b) manually mknod'ing the bare minimum device special
files needed to satisfy the build or (c) running the `udevstart'
program from Udev. All of these solutions are suboptimal in the
author's opinion which is why using MAKEDEV here is recommended.
What you do to manage /dev in the final deployed system is entirely
your choice, but if you're after the latest and greatest, Udev would
be your best bet.

7.9. Man-Pages-3.20

rm -fv man3/getspnam.3 man5/passwd.5make DESTDIR=$PM_DEST install

Rationale Notes

In general terms, GNU prefers the Info documentation system over man
pages. However, many GNU packages install both info pages and (in
some cases, less than adequate) man pages. The man pages listed
above will be installed later on by Coreutils, Diffutils and Shadow
and are therefore superfluous. We suppress their installation here
because some Package Managers will refuse to install an archive if a
file conflict is detected. If you'd prefer to keep the copies here,
omit the rm statements and somehow suppress the installation of said
man pages when installing the 3 aforementioned packages later on.

7.10. Man-Pages-Posix-2003-a

make DESTDIR=$PM_DEST install

7.11. Linux-Kernel-Headers-2.6.27

Warning

As mentioned earlier, the 2.6.18 kernel introduced the next generation
method of obtaining sanitized kernel headers. When using the new
technique you MUST SKIP the Linux-Libc-Headers
step which follows next.

7.13. Glibc (2.3.6 through 2.7)

Proceed with Caution!

Glibc is possibly the most critical piece of software you will compile
in a base Reference Build. There is a high probability of something
going wrong. Be sure to heed the advice contained in
Section B, “Glibc General Advice” before taking on this monster.

For an explanation of why we add `-mtune=generic' to CFLAGS in the
x86 GCC-4.2 and above case, please read
this
mailing list post.

Whenever Glibc is installed directly into a prefix of /usr via `make
install' (ie: the Makefile variable `install_root' is unset), a
script called `test-installation.pl' is run to perform some basic
sanity checks (compiling, linking, executing, etc) against the newly
installed Glibc. If all goes well, a message "Your new glibc
installation seems to be ok." will be shown. However, at the time
the script is run our toolchain defaults are still pointing to
$TT_PFX. In particular, ld's library search path is pointing to
$TT_PFX/lib and gcc's dynamic linker spec is pointing to
$TT_PFX${DL}. In actual fact, this means the checks are made against
the wrong Glibc. The problem will usually go unnoticed because the
Glibc's in $TT_PFX/lib and /usr/lib are functionally equivalent and
the checks always pass. You can expose the problem by building
different Glibc addons in each phase eg: configure the Temptools
phase Glibc with `--enable-add-ons=nptl' and configure the Chroot
phase Glibc with `--enable-add-ons=nptl,libidn' (assuming you have
downloaded and unpacked the separate addon for Libidn). We work
around the issue by using a sed to insert `-L/usr/lib
-Wl,-dynamic-linker=${DL}' at an appropriate place. Note that ld's
library search path and the gcc dynamic linker spec will be
readjusted to point to the new Glibc in the next step.

The sed to iconv/gconv_db.c is required to fix a bug that is known
to affect at least Samba (crash on startup). It essentially
replicates this
fix
from upstream. More information
here.

The sed to config.make on x86_64 fixes a subtle bug in the
autoconfigury which arrives at a wrong result for the "-z combreloc"
feature test. The issue is
fixed
in Glibc-2.4.x. More information
here.

The command to install the localedata has been commented out above.
If you need all the locales, unlikely as that may be, feel free to
uncomment the relevant line. It used to be the case that some of
these locales were required for the Libstdc++ testsuite to pass. But
that is no longer true with the advent of GCC-3.4. More information
about this issue is
here.
However, the Libstdc++ docs do recommend a minimum list of locales,
so having them installed should result in wider Libstdc++ testsuite
coverage. Note that you can install individual locales at any time
using the `localedef' command ie: locale installation doesn't have
to occur as part of the Glibc install. At the very least, you'll
probably want to install a locale for your own country.

We install the generic version of `bits/stdio-lock.h' into a
non-standard location because it gives us the capability of
pursuading some legacy software to compile successfully, in
particular, apps that define `_IO_MTSAFE_IO' eg: the libstdc+
library from GCC-2.95.x. Clearly, installing this header is a kludge
and of course optional. More information about the issue is
here.
Note that we use `find' to locate the header because it changed
location between Glibc-2.3.x and Glibc-2.4.x.

The warning in current LFS about unsetting CFLAGS and not overriding
the default optimization for Glibc is a crock. Glibc is EXACTLY the
kind of package that should be optimized - Refer Golden Rule No. 2
in Section B, “Glibc General Advice”. The warning in LFS dates back
to times when clues were scarce and tools weren't as good.

Tip -- Set up Dual Threading Libraries

Ever since the advent of NPTL most mainstream distros are set up with
both NPTL and Linuxthreads enabled Glibc's installed side by side.
This is for backwards compatibility reasons. NPTL is still relatively
new and a lot of commercial software simply won't run under it. Be
sure to check out Section B, “Dual Threading Libraries” if you need
this backwards compatibility and want to set up your system to match
the distros.

Note: if you're using GCC-3.4.4 or earlier and running under a 2.6.12
kernel or above, a new kernel feature known as "address space
randomization" will likely cause some PCH tests to fail as per these
unsatisfactory
results. The problem is now resolved in GCC-3.4.5 and above.
However, if for some reason you need to use an old GCC version, the
problem can be worked around by following the advice contained in
this
mailing list post.

When building with GCC-4.x we use a sed to add
`-fomit-frame-pointer' to the GCC Makefile to address a correctness
issue. Please read
this
mailing list post for some rationale discussion.

When building with GCC-4.x we use a sed to increase the timeouts on
2 libmudflap tests to avoid spurious failures. The problem is most
noticeable when running the GCC testsuite in parallel on SMP.
Upstream have been
notified of the
issue. Note the timeouts were increased from 3 to 10 seconds as of
GCC-4.1 but this is still not enough. The sed is compatible across
all supported GCC versions.

We add "MAKEINFO=makeinfo" and "gcc_cv_prog_makeinfo_modern=yes" to
the `make' invocation to ensure installation of the GCC info pages.
Please read
this
mailing list post for background details.

A newer `standards.info' is installed later by the Autoconf package.
Here we use `rm' and a Makefile sed to suppress installation of the
unwanted file. The `rm' is needed when using FSF releases but is
otherwise harmless when using HJL releases.

For an explanation of the Makefile.in sed, configure switches and
chown command, please read
this
mailing list post.

The sed to `tests/other-fs-tmpdir' adds /dev/shm as a location to
search when the testsuite goes looking for a writable directory on a
different disk partition. This allows some tests to run that would
otherwise be skipped. Passing `CANDIDATE_TMP_DIRS=/dev/shm' would
achieve the same effect but the sed saves us passing it twice.

A bunch of commands are moved to /bin as mandated by the FHS. In the
case of `[' and `test', the FHS says they must be together in either
/bin or /usr/bin. We leave them in /usr/bin to match the big
distros. Even though `sleep' and `touch' are not mentioned by the
FHS, we move them to /bin because: (a) they are commonly used in
bootscripts, (b) they have always traditionally been there and (c)
this matches the big distros. Please read
this
mailing list post for some rationale discussion.

We have opted to use the `su' from Coreutils rather than the one
from Shadow. Please read
this
mailing list post for some rationale discussion. If you
would rather use the `su' from Shadow, simply omit `su' from the
--enable-install-program configure switch and the 4th mv command.

If you would like `uname' to print something useful for the `-p' and
`-i' options instead of "unknown", 3rd party patches are known to
exist (or at least have existed in the past) at the
LFS
and
Gentoo
projects. But before applying any such hacks, ask yourself this
question: Why haven't these types of patches been integrated by the
upstream Coreutils developers? Some insight into this issue can be
found in
this
mailing list thread.

The sed to `configure' replicates a fix from upstream which ensures
the shared lib is always built with -fPIC. Currently, -fPIC is lost
if CFLAGS happen to be set in the environment. More information in
this
mailing list post. Note: Zlib beta versions can be found
here.

The reason for the mv -v {,x}libz.a etc. is
to allow better locality of the "installation" commands. More
rationale in
this
mailing list post.

7.21. Iana-Etc-2.30

make STRIP=yes
make testmake DESTDIR=$PM_DEST install

Rationale Notes

Note: You should be aware there are performance implications when
using this package because it results in an enormous /etc/services
file. Please read
this
mailing list post for some background discussion.

7.31. Flex-2.5.35

7.32. Gettext-0.17

./configure
make
make -k check || :make DESTDIR=$PM_DEST install

Rationale Notes

The Gettext testsuite takes a painfully long time to run, and seeing
as Gettext is not a critical component in the overall scheme of
things, running this testsuite is considered optional. You can
reduce the pain somewhat by only running the less time consuming
parts of the testsuite like so: make -C gettext-tools
check.

Note that the Gettext testsuite will gain wider coverage if you have
a bunch of various locales installed (the testsuite will inform you
of which ones are missing).

Note also that 2 tests will fail if Gettext is configured with
`--disable-nls' and the fr_FR locale is installed. More information
about this issue is
here.

7.41. Libtool-2.2.6a

7.42. Autoconf-2.63

7.43. Automake-1.10.1

./configure
make
make -k check || :make DESTDIR=$PM_DEST install

This release of Automake contains an expected testsuite failure when
run as root. More information
here.

7.44. Bash-3.2

Important

The Bash testsuite is unlike most others in that it doesn't stop if
errors are encountered. This means you have to physically examine the
output and cannot rely on the exit status of the `make tests' command.
It even tells you as much at the beginning of the run: "Any output
from any test, unless otherwise noted, indicates a possible anomaly".

Testing kernel module handling in a live kernel cannot be done
easily for obvious reasons. Therefore the Module-Init-Tools
developers have devised a custom testsuite procedure that involves
separately compiling the package with a set of stubs (see
testing.h). The net result of this means we must first run the
testsuite, perform a `make distclean' then compile the package for
real.

Please read the mailing list thread starting with
this
post for some rationale behind the above build commands.

The `bin_PROGRAMS=login suidbins=' addition to the `make install'
invocation is used to suppress installation of `groups' and `su'.
This is because we have opted to use the versions from Coreutils
instead. In the case of `su', please read
this
mailing list post for some rationale discussion. If you
would rather use the `su' from Shadow, replace the installation line
above with make DESTDIR=$PM_DEST bin_PROGRAMS="login su"
install. You would also want to adjust the sed to
`man/Makefile' to ensure installation of the su related man pages.

The sed to `stage2/size_test' is used to work around an issue with a
failing ufs2 test. Just ignoring the failure is unsatisfactory
because the `size_test' script bails out on the first failure and
therefore doesn't test the remaining file systems (fat, e2fs, minix)
2 of which are clearly important.

The "256 byte inode" patch could be considered optional in some
circumstances. But it's probably best to apply it. More detailed
rationale discussion can be found in
this
mailing list thread.

The Grub build is sensitive to non-standard CFLAGS in the
environment. Seeing as we have already set our CFLAGS globally in
`config.site', we force Grub to use its own defaults by configuring
with `default_CFLAGS=yes'.

7.67. Cpio-2.9 (Optional)

7.68. BPM-1.7 (Optional)

make DESTDIR=$PM_DEST install

A. Bootstrap Status

The DIY Linux Reference Build prides itself on being able to be
bootstrapped from many different build hosts (assuming, of course, a sane
toolchain). In essence, if the Reference Build fails to bootstrap from any
particular distro, we want to know about it so that the problem can be
addressed. This table details the current status of bootstraps from
various distributions that the author has tested personally. Expect the
number of entries to increase over time.

Distro Name

When Released

Glibc Version

GCC Version

Binutils Version

Kernel Version

Last Checked

Bootstrap Status

Debian 2.1

March 1999

2.0.7

2.7.2.3

2.9.1

2.0.38

FIXME

FAIL

Debian 2.2

August 2000

2.1.3

2.95.2

2.9.5

2.2.19

24 March 2005

OK

Debian 3.0

July 2002

2.2.5

2.95.4

2.12.90.0.1

2.2.20

26 March 2005

OK

Red Hat 6.2

March 2000

2.1.3

2.91.66

2.9.5

2.2.14

29 March 2005

OK

Fedora Core 3

November 2004

2.3.3

3.4.2

2.15.92.0.2

2.6.9

23 March 2005

OK

B. Toolchain Special Section

Glibc General Advice

Glibc is an incredibly complex beast. This is no "configure; make; make
install" type package. Extreme care is needed to get it right. If you're
not careful a build of Glibc will expose flaws in your:

Toolchain

Environment

Running Kernel

Kernel Headers

Technical Competence

Please read this enlightening post from the Glibc maintainer for the
"final word" on the subject:

Despite all of that, it is still possible to attain a DIY production Glibc
that works very well. Here are some golden rules for building Glibc:

Always compile and install in a chroot environment. At the very least,
build as a non-privileged user. Never `make install' over a live
system.

See how the pro's do it. Study the RPM spec files closely, especially
those from Red Hat/Fedora (Ulrich Drepper, Roland McGrath and Jakub
Jelinek are the main forces behind Glibc development and these guys
are all @redhat.com).

Stay abreast of toolchain issues. Monitor the mailing lists for
Binutils, GCC and of course Glibc. Also monitor the Kernel list if
possible.

Always run `make check'. Investigate and understand any failures.

Recognize that grabbing all the latest and greatest source packages
from ftp.gnu.org is never the best strategy for compiling Glibc.

Which Binutils?

FIXME - My text from the original Purelfs hint was integrated into the LFS
FAQ
here.
Take that text, revamp it, make it current, add the missing bits, then put
a new version here in the form of some rationale discussion.