Hi,
On Thu, Jan 17, 2013 at 08:33:57AM +0100, Wouter Verhelst wrote:
> If wanna-build is updated to support these two fields, then I imagine
> it can run the bootstrapping dependency algorithm. While you wouldn't
> want to upload a package to the debian.org archive when the
> architecture is as yet incomplete, the same isn't true for the
> debian-ports.org archive.
>
> It would require some patches for wanna-build to understand that package
> "foo" would need to be rebuilt once a particular profile is fully built
> and available, but I don't think that's impossible.
I'm not sure if wanna-build is the right tool to do this but we also did
not yet implement anything which would require a specific tool for the
build process.
The output of the current version of the algorithm produces is an ASCII,
comma separated list of source packages. All packages which can be built
in parallel are given on the same line. Source packages that need to be
built with reduced build dependencies are annotated as such. The
rebuilding of profile built packages is part of this output and
triggered after all source packages of a strongly connected component
finished building. After all profile built packages successfully rebuilt
fully, *all* source packages of the strongly connected component are
rebuilt. This is to make sure that every source package was built once
with all build dependencies available.
This list gave me so far an idea of the build order and a way to check
whether it made sense. By coming up with a more machine readable version
of this output, a tool can be triggered/controlled which then gives the
actual build jobs and does things like you mentioned above.
> However, to do that, there's one thing I'm missing in your mail: there
> will be cases where packages, when built in a particular profile do
> not support some functionality. That is, the package is available and
> does most of what the full package does, but because some
> build-dependency is missing, it doesn't support some feature or other
> -- for example, let's say that a document translation package (which
> we'll call "foo") which supports many input formats does not support
> XML as an input format when built in the stage1 profile. The output of
> its stage1 build would not change the number of binary packages built
> with the source package, it would just build a binary package with
> reduced functionality.
>
> Now it might be that a package build-depends on our package foo because
> it needs to translate documents in that XML format into something else,
> With your proposal, there's no way for the build-depending package to
> declare that it needs a version of foo that was built at a richer
> profile than stage1.
>
> Is this something you've considered?
Glad you bring this up because we indeed forgot to mention this in our
initial email. Yes, what you mention can indeed happen and this is also
why extensive rebuilding is part of the build order: to make sure that
all features are always compiled into the binaries.
We have thought about this issue but did not manage to come up with a
global solution. A solution would require a way to depend on "features"
of a package and therefor create an overcomplicated dependency syntax
which would not justify its purpose.
There are two things that a developer could do to avoid such FTBFS
errors during the bootstrap process when he modifies a source package
with reduced "stage1" build dependencies:
- only remove build dependencies which do not modify the binary
packages which are produced (instead, let some binary packages not be
produced, like foo-doc, foo-java or foo-gtk)
- look at the reverse dependencies in the generated dependency graph to
make sure that features which are changed in built binary packages
are not depended upon (naturally, those reverse dependencies will
change over time...)
Both solutions are very fragile. This is also why the developed
algorithms try to build as few source packages as possible with reduced
build dependencies.
Do we have some experience of how common such errors are during a manual
bootstrap?
cheers, josch