33.6. Construction of the Build Graph

During validation, abuild creates a
DependencyGraph object to represent the
space of build items and their dependencies. It performs a
topological sort on this graph to determine dependency order as
well as to detect errors and cycles in the dependency graph.
During the actual build, abuild needs to expand the dependency
graph to include not just build items but build item/platform
pairs. Every “instantiated” build item has to exist
on a particular platform. We refer to this platform-aware
dependency graph as the build graph. The
build graph can be inspected by running abuild with the
--dump-build-graph command-line option. For
information about the format of this output, see Appendix H, --dump-build-graph Format.

33.6.1. Validation

There are several steps required in order to determine exactly
which build items are to be built on which platforms and which
build item/platform pairs depend on which other pairs. Before
we do anything else, we need to perform several validations and
computations. The first of these is the determination of what
platform types a build item belongs to. For most build items,
this is simply the list of platform types declared in the build
item's Abuild.conf file. For build items
that have no build or interface files, there are no platform
types declared. In this case, the rules are different: if the
build item declares any dependencies and all of its directly
declared dependencies have identical platform type sets, then
the build item inherits its platform types from the items it
depends on. Otherwise, it has no platform types and has the
special target type all. Note that this analysis
is performed on build items in reverse dependency order (forward
build order). That way, every build item's platform types and
target type has been determined before any items that depend on
it are analyzed.

Once we have determined the list of platform types for each
build item, we can figure out which platforms a build item may
be built on. We refer to the list as the buildable
platform list. The buildable platform list for a
build item is included in the --dump-data
output (see Appendix F, --dump-data Format). Note that this is
generally a broader list than the list of platforms on which a
given build item will actually be built; the actual build
platform list is determined later in the build graph
construction process. For build items that have a specific
target type and platform types, the list of buildable platforms
is the union of all platforms supported on all platform types a
build item has. For items of target type all, we
don't explicitly compute a buildable platform list. These
platforms are allowed to “build” on any platform
since there are no actual build steps for such build items.
(Remember that for a build item to have target type
all, it must not have any declared platform types,
and this in turn means that it must have no build or interface
files.)

When we compute the buildable platform lists, we also
pre-initialize the build platform list (the list of platforms on
which the item will actually be built) by including all
buildable platforms that are selected by default on the basis of
any platform selectors, as described in Section 24.1, “Platform Selection”, that may be in effect. For
build items of target type all, we would not add
any items to the list at this step.

All of the above steps can be completed without knowing which
build items are actually included in the build set. These
computations, in fact, are determined at startup for every build
item in every known build tree regardless of whether the items
are in the build set.

The above validations are all completed before abuild starts
to build. If any errors are found in the above checks, abuild
will report them and exit before it attempts to construct the
build graph. This means that the build graph construction
itself can operate under the assumption that all of the above
constraints have been satisfied.

33.6.2. Construction

The next step is the construction of the actual build graph
itself. This is performed only when all previous validations
have been performed successfully, and this step is also
performed only for build items that are actually in the build
set. We present a prose description of the process here. For a
fully detailed description, please read the comments and code in
addItemToBuildGraph in
Abuild-build.cc (and in other places it
references) in the abuild sources.

We construct the build graph in reverse build order;
i.e., we start with least depended-on build
item and end with the most depended-on build item. That way,
each item is added to the build graph before any item it depends
on is added. This is the opposite of the order in which we
compute the platform types. This makes it possible to modify an
item's build platform list while processing items that depend on
it. Therefore, at the time a build item is added to the build
graph, its build platform list will have been fully computed.
The build platform list may be the initial list as computed
during validation, or it may have been modified during the
addition of its reverse dependencies to the build graph. When a
build item is added to the build graph, a node is added to the
build graph for each platform on which the item is being built.
Each node of the build graph therefore corresponds to a
build item/platform pair.
[65]

Then, for each direct dependency, we determine which instance of
it (which of its platforms) we will depend on for each of our
platforms. If the dependency in question is declared with a
platform selector, we pick the best platform from among the
dependency's buildable platform list that satisfies the platform
selector and make this the override
platform. If there are no matches, it is an error.
If an override platform is selected, it applies to this
dependency for all instances of the current item.

Next, still processing an individual dependency, we iterate
through the item's list of build platforms to decide which of
the dependency's platforms each instance will depend on. We
refer to this as the dependency platform. If we previously
computed an override platform for this dependency, we just use
that as the dependency platform. Otherwise, we pick the best
match from among the dependency's buildable platform list. If
the dependency is type all, it can be
“built” on any platform, so the dependency platform
is the current build platform of the item. If the dependency is
actually buildable on the exact platform that we are
considering, then it is the best match and the dependency
platform is the current platform. Otherwise, we have to search
for a platform from a compatible platform type. To do this, we
determine the platform type that contains the current platform
and then get a list of compatible platform types (as discussed
in Section 24.2, “Dependencies and Platform Compatibility”) in
order of preference. Then we iterate through this list until we
find a platform type that is in the dependency's list of
platform types. Once we have identified this type, we find the
best matching platform in that type and use that as the
dependency platform. The best matching platform will be first
selected platform, or if no platforms are selected, then it will
be the first unselected platform. If no platforms are
available, it is an error.

If we have successfully determined a dependency platform from
among the dependency's buildable platform list, we next add that
to the dependency's actual build platform list if it's not
already there. This is the mechanism by which as-needed
platform selection occurs. An example of this is presented in
Section 24.5, “Cross-Platform Dependency Example”. So if
item A on p1 wants item
B on p2, then item
B will be built on p2 even
if p2 would not have been selected to build for
B based on platform selectors. There are
many ways in which this can happen including
B being in a different build tree with
different plugins or A using a
platform-specific dependency to depend on
B.

33.6.3. Implications

Even if the exact steps of constructing the build graph are
involved, there are some implications that are worth separate
discussion. Specifically, a build item of target type
all may depend on any build item, and any build
item may depend on an item of target type all. For
other build items, if a build item depends on another build item
and declares the dependency with a
-platform=selector
option, the dependency must have the platform type mentioned in
the platform selector. If a build item depends on another item
without a platform-specific dependency, the dependency must be
buildable on at least one platform type that is compatible with
(or exactly matches) each platform type of the depending build
item. Since all platform types are compatible with
indep, this means that any build item may depend on
any other build item whose target type is
platform-independent. (This was actually a special
case prior to abuild 1.1.4, but now it falls out from the fact
that indep is made the parent platform type of all
platform types that don't declare a parent.) For example, if
A has platform types X and
Y and depends on B which has
types X, Y, and Z, this
is okay because B has all of
A's platform types. Likewise, if
platform types X and Y both declared
type XY as a parent and B
has types XY and Z, that would also be
fine since each of A's types can be
matched with at least one of B's types.
It would be an error if B depended on
A in either case since the instances of
B building for platform type
Z would not be able to satisfy their dependences on
A since it doesn't support that platform
type or anything compatible with that platform type.

Another important point involves the exact way in which we
search for a compatible platform. Note that we first search for
a compatible platform type and, once we have found one, we pick
the best platform in that type. This is subtly different from
picking the first matching platform from any compatible platform
type. To illustrate the difference, consider the case of
A, which has platform type
X, depending on B, which has
platform types Y and Z, where
X declares Y as a parent and
Y declares Z as a parent. In this
case, A on X will
always depend on B on
Y even if Y has no
platforms and Z does. The reason for
this is that platform types are static but platforms within
types are influenced by the environment. If Y has
no platforms, this should result in a build error. If we fell
back to Z, instead the lack of platforms
for Y would actually change the shape of
the build graph, which goes against abuild's design. If it
were specifically desired to fall back on one thing if something
else weren't available, there are ways to make that happen
(using autoconfiguration or other similar mechanisms) that don't
require the actual build graph itself to change shape.

There are also some implications of the way pass-through items
(items of type all) ignore dependencies when no
matching platform is available. This is discussed in Section 24.4, “Dependencies and Pass-through Build Items”. Specifically, if a
pass-through item depends on multiple items of different types,
it's possible for the pass-through to effectively connect one of
its reverse dependencies to multiple of its forward
dependencies. So if A depends on
P which depends on one item of type
indep and one item of A's
platform type, A will end up with
both of P's
dependencies in its dependency list. This is a consequence of
the fact that the dependency platform computation is performed
separately for each build platform and for each dependency of a
given item.

[65]
The actual build graph node is a string made up of three
fields: a fixed-width numeric prefix representing the build
tree, the item name, and the platform name. Numeric prefixes
for the trees are assigned based on a topological sort of the
tree dependency graph. When build graph nodes are sorted
lexically, the result is a topologically sorted list of trees
each containing a lexically sorted list of items in the tree.
This is the mechanism that abuild uses to build items in
less dependent trees before items in more dependent trees.