Buildlink is a framework in pkgsrc that controls what headers and libraries
are seen by a package's configure and build processes. This is implemented
in a two step process:

Symlink headers and libraries for dependencies into
BUILDLINK_DIR, which by default is a subdirectory
of WRKDIR.

Create wrapper scripts that are used in place of the normal compiler
tools that translate -I${LOCALBASE}/include and
-L${LOCALBASE}/lib into references to
BUILDLINK_DIR. The wrapper scripts also make
native compiler on some operating systems look like GCC, so that
packages that expect GCC won't require modifications to build with
those native compilers.

This normalizes the environment in which a package is built so that the
package may be built consistently despite what other software may be
installed. Please note that the normal system header and library paths,
e.g. /usr/include,
/usr/lib, etc., are always searched -- buildlink3 is
designed to insulate the package build from non-system-supplied
software.

14.1. Converting packages to use buildlink3

The process of converting packages to use the buildlink3
framework (“bl3ifying”) is fairly straightforward.
The things to keep in mind are:

Ensure that the build always calls the wrapper scripts
instead of the actual toolchain. Some packages are tricky,
and the only way to know for sure is the check
${WRKDIR}/.work.log to see if the
wrappers are being invoked.

Don't override PREFIX from within
the package Makefile, e.g. Java VMs, standalone shells,
etc., because the code to symlink files into
${BUILDLINK_DIR} looks for files
relative to “pkg_info -qp pkgname”.

Remember that only the
buildlink3.mk files that you list in a
package's Makefile are added as dependencies for that package.

If a dependency on a particular package is required for its libraries and
headers, then we replace:

DEPENDS+= foo>=1.1.0:../../category/foo

with

.include "../../category/foo/buildlink3.mk"

The buildlink3.mk files usually define the required dependencies.
If you need a newer version of the dependency when using buildlink3.mk
files, then you can define it in your Makefile; for example:

There are several buildlink3.mk
files in pkgsrc/mk
that handle special package issues:

bdb.buildlink3.mk chooses either
the native or a pkgsrc Berkeley DB implementation based on
the values of BDB_ACCEPTED and
BDB_DEFAULT.

curses.buildlink3.mk: If the system
comes with neither Curses nor NCurses, this will take care
to install the devel/ncurses package.

krb5.buildlink3.mk uses the value
of KRB5_ACCEPTED to choose between
adding a dependency on Heimdal or MIT-krb5 for packages that
require a Kerberos 5 implementation.

motif.buildlink3.mk checks for a
system-provided Motif installation or adds a dependency on
x11/lesstif, x11/motif or x11/openmotif. The user can set
MOTIF_TYPE to “dt”,
“lesstif”, “motif” or
“openmotif” to choose
which Motif version will be used.

readline.buildlink3.mk checks for a
system-provided GNU readline or editline (libedit) installation,
or adds a dependency on devel/readline,
devel/editline. The user can set
READLINE_DEFAULT to choose readline implementation.
If your package really needs GNU readline library, its Makefile
should include devel/readline/buildlink3.mk
instead of readline.buildlink3.mk.

oss.buildlink3.mk defines several
variables that may be used by packages that use the
Open Sound System (OSS) API.

pgsql.buildlink3.mk will accept
any of the Postgres versions in the variable
PGSQL_VERSIONS_ACCEPTED and default to
the version PGSQL_VERSION_DEFAULT. See
the file for more information.

pthread.buildlink3.mk uses the value of
PTHREAD_OPTS and checks for native pthreads or adds
a dependency on devel/pth as needed.

xaw.buildlink3.mk uses the value of
XAW_TYPE to choose a particular Athena widgets
library.

The comments in those buildlink3.mk
files provide a more complete
description of how to use them properly.

14.2. Writing buildlink3.mk files

A package's buildlink3.mk file is
included by Makefiles to indicate the need to compile and link
against header files and libraries provided by the package. A
buildlink3.mk file should always provide
enough information to add the correct type of dependency
relationship and include any other
buildlink3.mk files that it needs to find
headers and libraries that it needs in turn.

To generate an initial buildlink3.mk
file for further editing, Rene Hexel's pkgtools/createbuildlink
package is highly recommended. For most packages, the following
command will generate a good starting point for
buildlink3.mk files:

%cd pkgsrc/category/pkgdir% createbuildlink >buildlink3.mk

14.2.1. Anatomy of a buildlink3.mk file

The following real-life example
buildlink3.mk is taken
from pkgsrc/graphics/tiff:

The header and footer manipulate
BUILDLINK_TREE, which is common across all
buildlink3.mk files and is used to track
the dependency tree.

The main section is protected from multiple inclusion
and controls how the dependency on pkg is
added. Several important variables are set in the section:

BUILDLINK_API_DEPENDS.pkg
is the actual dependency recorded in the installed
package; this should always be set using
+= to ensure that
we're appending to any pre-existing list of values. This
variable should be set to the first version of the
package that had an backwards-incompatible API change.

BUILDLINK_PKGSRCDIR.pkg
is the location of the pkg
pkgsrc directory.

BUILDLINK_DEPMETHOD.pkg
(not shown above) controls whether we use
BUILD_DEPENDS or
DEPENDS to add the dependency on
pkg. The build dependency is
selected by setting
BUILDLINK_DEPMETHOD.pkg
to “build”. By default, the full dependency is
used.

BUILDLINK_INCDIRS.pkg
and
BUILDLINK_LIBDIRS.pkg
(not shown above) are lists of subdirectories of
${BUILDLINK_PREFIX.pkg}
to add to the header and library search paths. These
default to “include” and “lib”
respectively.

BUILDLINK_CPPFLAGS.pkg
(not shown above) is the list of preprocessor flags to add
to CPPFLAGS, which are passed on to the
configure and build phases. The “-I” option
should be avoided and instead be handled using
BUILDLINK_INCDIRS.pkg as
above.

The following variables are all optionally defined within
this second section (protected against multiple inclusion) and
control which package files are symlinked into
${BUILDLINK_DIR} and how their names are
transformed during the symlinking:

BUILDLINK_FILES.pkg
(not shown above) is a shell glob pattern relative to
${BUILDLINK_PREFIX.pkg}
to be symlinked into
${BUILDLINK_DIR},
e.g. include/*.h.

BUILDLINK_FILES_CMD.pkg
(not shown above) is a shell pipeline that
outputs to stdout a list of files relative to
${BUILDLINK_PREFIX.pkg}.
The resulting files are to be symlinked
into ${BUILDLINK_DIR}. By default,
this takes the +CONTENTS of a
pkg and filters it through
${BUILDLINK_CONTENTS_FILTER.pkg}.

BUILDLINK_CONTENTS_FILTER.pkg
(not shown above) is a filter command that filters
+CONTENTS input into a list of files
relative to
${BUILDLINK_PREFIX.pkg}
on stdout. By default,
BUILDLINK_CONTENTS_FILTER.pkg
outputs the contents of the include
and lib directories in the package
+CONTENTS.

BUILDLINK_FNAME_TRANSFORM.pkg
(not shown above) is a list of sed arguments used to
transform the name of the source filename into a
destination filename, e.g. -e
"s|/curses.h|/ncurses.h|g".

This section can additionally include any
buildlink3.mk needed for
pkg's library dependencies.
Including these buildlink3.mk files
means that the headers and libraries for these
dependencies are also symlinked into
${BUILDLINK_DIR}
whenever the pkgbuildlink3.mk
file is included. Dependencies are only added for directly
include buildlink3.mk files.

When providing a buildlink3.mk and
including other buildlink3.mk files in it,
please only add necessary ones, i.e., those whose libraries or
header files are automatically exposed when the package is
use.

In particular, if only an executable
(bin/foo) is linked against a library, that
library does not need to be propagated in the
buildlink3.mk file.

The following steps should help you decide if a
buildlink3.mk file needs to be included:

Look at the installed header files: What
headers do they include? The packages providing these files
must be buildlinked.

Run ldd on all installed
libraries and look against what other libraries they link.
Some of the packages providing these probably need to be
buildlinked; however, it's not automatic, since e.g. GTK on
some systems pulls in the X libraries, so they will show up in
the ldd output, while on others (like OS
X) it won't. ldd output can thus only be
used as a hint.

These two variables differ in that one describes source
compatibility (API) and the other binary compatibility (ABI).
The difference is that a change in the API breaks compilation of
programs while changes in the ABI stop compiled programs from
running.

Changes to the
BUILDLINK_API_DEPENDS.pkg
variable in a buildlink3.mk file happen
very rarely. One possible reason is that all packages depending
on this already need a newer version. In case it is bumped see
the description below.

The most common example of an ABI change is that the major
version of a shared library is increased. In this case,
BUILDLINK_ABI_DEPENDS.pkg
should be adjusted to require at least the new package version.
Then the packages that depend on this package need their
PKGREVISIONs increased and, if they have
buildlink3.mk files, their
BUILDLINK_ABI_DEPENDS.pkg
adjusted, too. This is needed so pkgsrc will require the correct
package dependency and not settle for an older one when building
the source.

Please take careful consideration before adjusting
BUILDLINK_API_DEPENDS.pkg
or
BUILDLINK_ABI_DEPENDS.pkg
as we don't want to cause unneeded package deletions and
rebuilds. In many cases, new versions of packages work just
fine with older dependencies.

Also it is not needed to set
BUILDLINK_ABI_DEPENDS.pkg
when it is identical to
BUILDLINK_API_DEPENDS.pkg.

14.3. Writing builtin.mk files

Some packages in pkgsrc install headers and libraries that
coincide with headers and libraries present in the base system.
Aside from a buildlink3.mk file, these
packages should also include a builtin.mk
file that includes the necessary checks to decide whether using
the built-in software or the pkgsrc software is
appropriate.

The only requirements of a builtin.mk file for
pkg are:

It should set
USE_BUILTIN.pkg
to either “yes” or “no”
after it is included.

It should not override any
USE_BUILTIN.pkg
which is already set before the
builtin.mk file is included.

It should be written to allow multiple inclusion. This
is very important and takes careful
attention to Makefile coding.

The first section sets
IS_BUILTIN.pkg
depending on if pkg really exists
in the base system. This should not be a base system software
with similar functionality to pkg;
it should only be “yes” if the actual package is
included as part of the base system. This variable is only
used internally within the builtin.mk
file.

The second section sets
BUILTIN_PKG.pkg
to the version of pkg in the base
system if it exists (if
IS_BUILTIN.pkg
is “yes”). This variable is only used internally
within the builtin.mk file.

The third section sets
USE_BUILTIN.pkg
and is required in all
builtin.mk files. The code in this
section must make the determination whether the built-in
software is adequate to satisfy the dependencies listed in
BUILDLINK_API_DEPENDS.pkg.
This is typically done by comparing
BUILTIN_PKG.pkg
against each of the dependencies in
BUILDLINK_API_DEPENDS.pkg.
USE_BUILTIN.pkgmust be set to the correct value by the
end of the builtin.mk file. Note that
USE_BUILTIN.pkg
may be “yes” even if
IS_BUILTIN.pkg
is “no” because we may make the determination
that the built-in version of the software is similar enough to
be used as a replacement.

The last section is guarded by
CHECK_BUILTIN.pkg,
and includes code that uses the value of
USE_BUILTIN.pkg
set in the previous section. This typically includes, e.g.,
adding additional dependency restrictions and listing additional
files to symlink into ${BUILDLINK_DIR} (via
BUILDLINK_FILES.pkg).

14.3.2. Global preferences for native or pkgsrc software

When building packages, it's possible to choose whether to set
a global preference for using either the built-in (native)
version or the pkgsrc version of software to satisfy a
dependency. This is controlled by setting
PREFER_PKGSRC and
PREFER_NATIVE. These variables take values
of either “yes”, “no”, or a list of
packages. PREFER_PKGSRC tells pkgsrc to
use the pkgsrc versions of software, while
PREFER_NATIVE tells pkgsrc to use the
built-in versions. Preferences are determined by the most
specific instance of the package in either
PREFER_PKGSRC or
PREFER_NATIVE. If a package is specified
in neither or in both variables, then
PREFER_PKGSRC has precedence over
PREFER_NATIVE. For example, to require
using pkgsrc versions of software for all but the most basic
bits on a NetBSD system, you can set:

PREFER_PKGSRC= yes
PREFER_NATIVE= getopt skey tcp_wrappers

A package must have a
builtin.mk
file to be listed in PREFER_NATIVE,
otherwise it is simply ignored in that list.