Hi,
(I was not able to find the debian-ports list on lists.debian.org (so I
subscribed via email) did I just miss it?)
Quoting Steven Chamberlain (2013-10-23 22:04:59)
> I had a play with the 'botch' tool (see description[1]) for determining build
> order when bootstrapping an architecture.
botch author here. Just stumbled upon this already a few day old email in my
inbox :)
> To start off with it determines a minimum required set of packages - you'd
> normally cross-compile those from another system. This set (see attached
> example list for kfreebsd-amd64 wheezy) looks to me like what constitutes the
> 'toolchain'.
This minimum set of packages which has to be cross compiled (because no binary
package of the target architecture exists at this point) is what we call the
minimal native build system (the name is misleading as disjunct dependencies
make different choices of this set possible).
Currently it is not possible to present a correct selection of source packages
which have to be cross compiled to produce the minimal build system. What we
currently do is to just do:
grep-dctrl -X \( -FPackage build-essential --or -FPackage debhelper --or -FEssential yes)
and assume that the resulting list of packages (the one you attached to your
last email) is cross compilable from nothing. This is of course not the case in
practice but a formal analysis is not possible yet. This is because multiarch
annotations are missing in some packages and because of the problem of how to
handle source packages directly depending on gcc, g++, binutils etc in the
cross compilation case. While the first one is easy to fix there doesnt exist a
solution for the second one yet. Build profiles would be able to solve the
second problem.
Until these two issues are fixed we will not be able to get an algorithmic
answer to the question of what constitutes the minimum required set of
packages.
On the other hand wookey did lots of ubuntu crossing and identified that with
just (I think it was) a dozen modified packages (reducing their build
dependencies to break cross build dependency cycles) he was able to cross build
all of these packages:
http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html
So while an automated analysis is not possible right now, it also does not seem
to be necessary to have this automated. Having the to-be-crossed selection of
packages retrieved automatically becomes more interesting as more source
packages are known to be cross-compilable including all their required
recursive build dependencies.
> The list will be different for each port, and change over time. This example
> included freebsd-libs, freebsd-utils and kfreebsd-kernel-headers but of
> course others won't.
Thanks for trying out botch for kfreebsd :)
> AIUI those packages should be able to rebuild each of themselves without any
> other dependencies.
Should but unfortunately they are not :(
In fact, to nativel rebuild the minimal build system for amd64 (just
essential:yes + build-essential + debhelper) one needs to be able to compile
383 source packages (excluding the source packages in the minimal build system
itself).
This is as of debconf13 when I last ran those scripts. You can look at the
numbers here as well:
http://mister-muffin.de/bootstrap/stats/
These 383 source packages include ugly ones like iceweasel, nautilus, webkit,
network-manager, mysql, kde4libs which as you can imagine draw in half of what
makes a modern desktop system and thus might naively have been dismissed as
non-essential for the bootstrapping purpose at all. But of course this list
will be different between arches.
> I think doing that regularly would be a good health check for a port's
> toolchain. Probably these packages would be the focus of the
> reproducible-builds project, at least from a security point-of-view. Any
> differences in output of subsequent builds are of interest, and would
> potentially reveal when significant changes or bugs were introduced too.
Being able to use botch to automatically bootstrap all arches from scratch has
always been one of botch's goals. Unfortunately the build profile discussion is
holding up the implementation of this in practice but guillem promised to look
into this for dpkg 1.17.2.
cheers, josch