>Number: 35012
>Category: pkg
>Synopsis: pkgsrc binary package dependancy system is broken
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: pkg-manager
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Nov 08 02:30:00 +0000 2006
>Originator: Robert Elz
>Release: NetBSD 3.99.15 (pkgsrc current as of last few hours)
>Organization:
Prince of Songkla University
>Environment:
System: NetBSD jade.coe.psu.ac.th 3.99.15 NetBSD 3.99.15 (GENERIC-1.696-20060125) #8: Wed Jan 25 04:59:39 ICT 2006 kre@jade.coe.psu.ac.th:/usr/obj/current/kernels/JADE_ASUS i386
Architecture: i386
Machine: i386
>Description:
Binary packages are built with a copy of the source package
dependencies as their dependencies. That's totally broken.
A binary package *must* depend upon at least the version of
another package that was installed at the time it was built, not
the minimum version that might have been installed in order to
build the source package.
To see this, just consider the NetBSD C library (I know this is
not a pkgsrc binary, but it illustrates the problem). The vast
majority of NetBSD programs will compile, link and run, against
any version of the NetBSD C library (that is, the source dependency
is simply that a C library exists). Those programs that need at
least a particular version - eg: maybe they need statvfs or something,
show the same issue, but we don't need to consider those.
That is, for example, /usr/src/bin/cat/* will happily compile
and link (I guess, if I happened to pick a bad example, just
pick another) against any version of the C library.
However, a version that has been linked with the NetBSD 3 version
of the C library isn't really expected to work on NetBSD 1.6, or
NetBSD 2 (it might, sometimes, if it is lucky, but it isn't really
expected). On the other hand, a version linked against the
NetBSD 3 C library is expected to work on a NetBSD 4 system.
pkgsrc dependency system simply ignores this, and claims that
any version will do, when it builds a binary package. That's bogus.
>How-To-Repeat:
The way I (am fairly sure) I saw this just recently was by
installing firefox2, to see what it was like. First, I was most
pleased to see I could install it without having to delete and
upgrade half the universe first, I hadn't really expected that,
even better, that it installed wthout even needing to remove
firefox1 (or its gtk1 version) first. Credit to the firefox
package developers for that.
However, it really didn't work for me at all once installed,
some functionality was there, but core dumps, ... I haven't
bothered to debug this, but I think the problem is that I always
install from binary packages (and did this time). I build
them myself, but I always build the binary package first, and
then install it.
The binary package I built for firefox2 claims that it needs...
jpeg>=6bnb2
libIDL>=0.8.6nb1
gtk2+>=2.8.17nb1
cairo>=1.0.4nb1
png>=1.2.9nb2
Xft2>=2.1.7nb2
And on the system I installed it, what I have installed is ...
jpeg-6bnb3 IJG's jpeg compression utilities
libIDL-0.8.7 CORBA Interface Definition Language parser
gtk2+-2.8.20nb1 GIMP Toolkit v2 - libraries for building X11 user interfaces
cairo-1.2.0nb2 Vector graphics library with cross-device output support
png-1.2.12 Library for manipulating PNG images
Xft2-2.1.7nb2 Library for configuring and customizing font access
which is ever dependency satisfied, nothing needed upgrading
to install firefox2.
However, now let's look at the packages that were installed
when the firefox2 binary package was built - first, this is
it (and its build date)
-rw-r--r-- 1 root wheel 14789895 Nov 1 01:38 firefox2-2.0.tgz
And the dependency binary packages (which are all what were
installed in my pkg_comp arena when firefox2 was compiled)
-rw-r--r-- 1 root wheel 305432 Jul 19 16:39 jpeg-6bnb3.tgz
-rw-r--r-- 1 root wheel 154330 Aug 6 00:15 libIDL-0.8.7.tgz
-rw-r--r-- 1 root wheel 11155929 Oct 15 22:56 gtk2+-2.10.6.tgz
-rw-r--r-- 1 root wheel 529259 Sep 30 19:42 cairo-1.2.4nb3.tgz
-rw-r--r-- 1 root wheel 307696 Jul 19 16:32 png-1.2.12.tgz
-rw-r--r-- 1 root wheel 78732 Sep 18 22:36 Xft2-2.1.7nb2.tgz
Some of those are fine (ones that have not been upgraded in
a while, like jpeg libIDL png and Xft2)
But, firefox2 was compiled against gtk2+ version 2.10.6
and is not attempting to use version 2.8.20 to run
(which pkg_add doesn't object to, as the package only
claims that it needs 2.8.17 or better). Similarly,
cario version 1.2.4 (the nb3 most likely doesn't matter)
was used to build firefix2, but version 1.2.0 is being
used to run it (again, pkg_add has no idea, as all it is
being told is that anything 1.0.4nb1 or later is OK).
I'm not going to go and attempt to figure out why firefox2
was behaving strangely with a setup like this - I wouldn't
expect it to work properly, and I am not surprised in the
least that it doesn't. Nor is the problem with pkg_add,
as fas as it could tell, everytyhing was fine. The problem
is in pkgsrc, with what dependency information it is putting
in the binary packages when it makes them.
Note that this is what *any* user of binary packages will
experience - note that all of the available binary packages are
consistent, if I removed and reinstalled all of my /usr/pkg
(or a substantial subset of it) this problem wouldn't arise.
That is, it isn't a problem of building one binary package
that's out of step with everything else. There's no way that
you can rationally expect anyone to do that however - rmeove and
reinstall everything, every time they add a new binary package.
And *everything* is th eonly choice most people are likely to
have, as there's no current way to tell them which of their
installed packages do need upgrading (I guess they could
upgrade everything that is out of date - for me, that would
have meant upgrading bind9 and bash and ... even though none of
those has anything to do with firefox at all - I *think*).
And even that assumes that there's a rational way to figure out
what packages I have installed are out of date, if I have no
pkgsrc on my system (relying entirely on binary packages) and
the binary packages are fetched over the net, what technique would
someone use to work out which of their installed packages is
out of date (even if this was the right way forward - which it
is not).
>Fix:
When a binary package is built, the dependencies (for at least
libraries, and anything else that influences the compilation
process - anything that provides C .g files, or their equivalents
in other languages) MUST be (at least) the versions that were
installed when the binary package was built - later versions
are generally OK (most library developers attempt to keep backwards
binary compatability - not all of them, but most - even more than
backwards source compatability).
In this case, that would have told me that I needed
gtk2+>=2.10.6
(etc) and then i would have known that I needed to upgrade
gtk2+ before firefox2 could be installed (which most likely
would have meant upgrading a bunch of other stuff as well).
Note: this cannot (should not, or even must not) be fixed with
the dependencies in the source versions of the packages.
If firefox2 says that gtk2+ version 2.8.17nb1 or better is
OK to use to compile against, then that's all that should be
required - that is, if I compiled firefox2 on the system
I installed it on, upgrading nothing, I'd expect it to work.
(If that's not true, then there's a bug in the firefox2
dependency list, but that would, if it happened to be true,
be a comparatively trivial issue - and I have no reason at all
to assume that anything like that is a problem).