Tue, 25 Mar 2008 - pkg-config, sonames and Requires.private

This post is both an attempt at replying to a bug against
telepathy-glib, but also an attempt at explaining what
Requires.private do (and don't).

I am using Evolution as my example here, not to pick on Evolution or its
authors in any way, but because it's a convenient example. Currently,
on Ubuntu Hardy, evolution links against 75 different libraries.
Amongst those, we find libz.so.1, libXinerama.so.1 and many more.
I'll go out on a limb here and claim that Evolution does not call any of
the functions in libXinerama directly. Let that be the assumption from
here on.

An obvious question then is, why does evolution link against
libXinerama.so.1 if it doesn't use it? To answer that question, we
need to go back in time to before we had dynamic linking. If you wanted
to build a binary like evolution you had to have 75 -l statements
when you linked and you ended up with the whole code for Xinerama
embedded in your email and calendar client. For various reasons, we
stopped doing that and switched to dynamic linking where the evolution
binary just contains a reference to libXinerama. At some point we
also grew the ability for libraries to contain those references to other
libraries, so you don't have to hunt down all the dependencies of
libfoo when you are linking with it. We also got tools such as
libtool which try to abstract away a lot of the problems of building
on older platforms which don't support inter-library dependencies.

Now, since evolution still doesn't use anything directly in
libXinerama.so.1 but just uses a library which in turn links against
libXinerama.so.1, it shouldn't be linking against it. Then why is it
linked with it? Again, we need to look back at history, and for this
part I am at least partially responsible.

pkg-config was originally written as a replacement for gnome-config
and various other -config utilities. Lots of libraries and
applications now ship .pc files and we have a standardised interface
for querying those files. One of the problems the original authors of
pkg-config faced was the problem of dependencies. They added
dependencies so the authors of gst-python-0.10 could say "We need
pygtk-2.0 too" and so the compilation flags needed for gst-python-0.10
would also include those for pygtk-2.0. Note that I'm using
"compilation flags" loosely here, I am not just talking about CFLAGS.

This did not fix the problem of inflated dependencies. Not at all. I
talked with some of the Debian Release Managers back in 2004/2005 and
we worked out a solution which should help us have correct, uninflated
dependencies since the then-current way of handling dependencies caused
big problems for migrations of packages between unstable and
testing.

The plan was to introduce a new field, Requires.private which would
not show up unless you passed --static to the pkg-config invocation
(since you need all libraries if you are linking statically). This
definition of Requires.private was mostly useless since GNOME and GTK+
have a habit of including each other's headers. To make a long story
short, I changed the semantics so the Cflags field from private
dependencies were included even when not linking statically.

A problem which pkg-config does nothing to guard against in this case
is if you have libfoo.so.2 linking against libbar.so.1 and
libfoo.so.2 exports some of libbar's types in its ABI (and not just as
pointers, but actual structs and such). If libbar's soname is then
bumped to libbar.so.2 and libfoo is rebuilt, libfoo's ABI has changed
without a soname bump. This is bad and will cause problems. If your
application is linked against both libfoo.so.2 and libbar.so.1,
you'll still get problems since libfoo.so.2 then suddenly pulls in
symbols from libbar.so.2. If you used symbol versioning, you would at
least not get symbol conflicts and your application would continue to
work, but you would have a spurious dependency and the package
containing libbar.so.1 would be kept around until your application was
recompiled.

With this background, you might ask the question why we still have
Requires since it is seemingly useless. For C, it is useless in all
but the most special cases, just use Requires.private instead (and its
sibling Libs.private). Other languages have different semantics.
Some people use .pc files for other purposes such as
gnome-screensaver having variables defining where themes and
screensavers go.

Hopefully this blog post has explained a bit about why we have
Requires.private and what the difference between this and their
regular counterpart is. If there's anything unclear, please do not
hesitate to contact me.