Chapter 5 Application Binary Interfaces and Versioning

ELF objects processed by the link-editors provide many global symbols to which other objects can bind.
These symbols describe the object's application binary interface (ABI). During the evolution of an object, this interface can change due to the addition or deletion of global
symbols. In addition, the object's evolution can involve internal implementation changes.

Versioning refers to several techniques that can be applied to an object to indicate interface and implementation changes.
These techniques provide for controlled evolution of the object, while maintaining backward compatibility.

This chapter describes how to defined an object's ABI. Also covered, are how changes to this ABI interface can affect backward compatibility. These concepts
are explored with models that convey how interface, together with implementation changes, can be incorporated into a new release of an object.

The focus of this chapter is on the runtime interfaces of dynamic executables and shared objects. The techniques used to describe and manage changes within these dynamic objects are presented
in generic terms. A common set of naming conventions and versioning scenarios as applied to shared objects can be found in Appendix B, Versioning Quick Reference.

Developers of dynamic objects must be aware of the ramifications of an interface change and understand how such changes can be managed, especially in regards to maintaining backward compatibility
with previously shipped objects.

The
global symbols that are made available by any dynamic object represent the object's public interface. Frequently, the number of global symbols that remain in an object after a link-edit are more
than you would like to make public. These global symbols stem from the symbol state that is required between the relocatable objects used to create the object. These symbols represent private interfaces
within the object.

To define an object'sABI, you should first determine those global symbols that you want to make publicly available from the object. These public symbols can be established
using the link-editor's -M option and an associated mapfile as part of the final link-edit. This technique is introduced in Reducing Symbol Scope. This public interface establishes one or more version definitions within the object being created. These definitions form the foundation for the addition of new interfaces
as the object evolves.

The following sections build upon this initial public interface. First though, you should understand how various changes to an interface can be categorized so that these interfaces can be
managed appropriately.

Interface Compatibility

Many types of change can be made to an object. In their simplest terms, these changes can be categorized into one of two groups.

Compatible updates. These updates are additive. All previously available interfaces remain intact.

Incompatible updates. These updates change the existing interface. Existing users of the interface can fail, or behave incorrectly.

The change of size, or content, of a data item to a function or as an external definition

Incompatible

A bug fix, or internal enhancement to a function, providing the semantic properties of the object remain unchanged

Compatible

A bug fix, or internal enhancement to a function when the semantic properties of the object change

Incompatible

Note –

Because of interposition, the addition of a symbol can constitute an incompatible update.
The new symbol might conflict with an applications use of that symbol. However, this form of incompatibility does seem rare in practice as source-level name space management is commonly used.

Compatible updates can be accommodated by maintaining version definitions that are internal to the object being generated. Incompatible updates can be accommodated by producing a new
object with a new external versioned name. Both of these versioning techniques enable the selective binding of applications. These techniques also enable verification of correct version binding
at runtime. These two techniques are explored in more detail in the following sections.

Internal Versioning

A dynamic object can have one or more internal version definitions associated with
the object. Each version definition is commonly associated with one or more symbol names. A symbol name can only be associated with one version definition. However, a version
definition can inherit the symbols from other version definitions. Thus, a structure exists to define one or more independent, or related, version definitions within the object being created. As
new changes are made to the object, new version definitions can be added to express these changes.

Version definitions within a shared object provide two capabilities.

Dynamic objects that are built against a versioned shared object can record their dependency on the version definitions bound to. These version dependencies are verified at runtime
to ensure that the appropriate interfaces, or functionality, are available for the correct execution of an application.

Dynamic objects can select the version definitions of a shared object to bind to during their link-edit. This mechanism enables developers to control their dependency on a shared
object to the interfaces, or functionality, that provide the most flexibility.

Creating a Version Definition

Version definitions commonly consist of an association of symbol names to a unique version name. These associations are established within a mapfile and supplied to the final link-edit of an object using the link-editor's -M option. This technique is introduced in the section Reducing Symbol Scope.

A version definition is established whenever a version name is specified as part of the mapfile directive. In the following example, two source files are combined, together
with mapfile directives, to produce an object with a defined public interface.

The symbol foo1 is the only global symbol that is defined to provide
the shared object's public interface. The special auto-reduction directive “*” causes the reduction of all other global symbols to have local binding within the object being generated.
The auto-reduction directive is introduced in Defining Additional Symbols with a mapfile. The associated version name, SUNW_1.1, causes the generation
of a version definition. Thus, the shared object's public interface consists of the global symbol foo1 associated to the internal version definition SUNW_1.1.

Whenever a version definition, or the auto-reduction directive, are used
to generate an object, a base version definition is also created. This base version is defined using the name of the object being built. This base version is used to associate any reserved symbols
generated by the link-editor. See Generating the Output File for a list of reserved symbols.

The version definitions that are contained within an object can be displayed using pvs(1) with the -d option.

$ pvs -d libfoo.so.1
libfoo.so.1;
SUNW_1.1;

The object libfoo.so.1 has an internal version definition named SUNW_1.1, together with a base version definition libfoo.so.1.

Note –

The link-editor's -z noversion option allows symbol reduction
to be directed by a mapfile but suppresses the creation of version definitions.

From this initial version definition, the object can evolve by adding new interfaces together with updated functionality. For example, a new function, foo2, together with
its supporting data structures, can be added to the object by updating the source files foo.c and data.c.

A new version definition, SUNW_1.2, can be created to define a new interface representing the symbol foo2. In addition, this new interface can be defined
to inherit the original version definition SUNW_1.1.

The creation of this new interface is important, as the interface describes the evolution of the object. These interfaces enable users to verify and select the interfaces to bind with. These
concepts are covered in more detail in Binding to a Version Definition and in Specifying a Version Binding.

The following example shows the mapfile directives that create these two interfaces.

The symbols foo1 and foo2 are both defined to be part of the shared object's public interface. However, each of these symbols is assigned to a different
version definition. foo1 is assigned to version SUNW_1.1. foo2 is assigned to version SUNW_1.2.

These version definitions, their inheritance, and their symbol association can be displayed using pvs(1) together with the -d, -v and -s options.

The version definition SUNW_1.2 has a dependency on the version definition SUNW_1.1.

The inheritance of one version definition by another version definition is a useful technique. This inheritance reduces the version information that is eventually recorded by any object that
binds to a version dependency. Version inheritance is covered in more detail in the section Binding to a Version Definition.

A version definition symbol is created and associated with a version definition. As shown in the previous pvs(1) example, these symbols are displayed when using the -v option.

Creating a Weak Version Definition

Internal changes to an object that do not require the introduction of a new interface definition can be defined by creating a weak version definition. Examples of such
changes are bug fixes or performance improvements. Such a version definition is empty. The version definition has no global interface symbols associated with the definition.

For example, suppose the data file data.c, used in the previous examples, is updated to provide more detailed string definitions.

The empty version definition is signified by the weak label. These weak version definitions enable applications to verify the existence of a particular implementation detail. An application
can bind to the version definition that is associated with an implementation detail that the application requires. The section Binding to a Version Definition illustrates
how these definitions can be used in more detail.

Defining Unrelated Interfaces

The previous examples show how new version definitions added to an object inherit any existing version definitions. You can also create version definitions that are unique and independent.
In the following example, two new files, bar1.c and bar2.c, are added to the object libfoo.so.1. These files contribute two new symbols, bar1 and bar2, respectively.

These two symbols are intended to define two new public interfaces. Neither of these new interfaces are related to each other. However, each interface expresses a dependency on the original SUNW_1.2 interface.

Version definitions can be used to verify runtime binding requirements. Version definitions can also be used to control the binding of an object during the objects creation. The following
sections explore these version definition usages in more detail.

Binding to a Version Definition

When a dynamic executable or shared object is built against other shared objects, these dependencies are recorded in the resulting object. See Shared Object Processing and Recording a Shared Object Name for more details. If a dependency also contain version definitions, then an associated version
dependency is recorded in the object being built.

The following example uses the data files from the previous section to generate a shared object, libfoo.so.1, which is suitable for a compile time environment.

Six public interfaces are offered by the shared object libfoo.so.1. Four of these interfaces, SUNW_1.1, SUNW_1.2, SUNW_1.3a,
and SUNW_1.3b, define exported symbol names. One interface, SUNW_1.2.1, describes an internal implementation change to the object. One interface, libfoo.so.1, defines several reserved labels. Dynamic objects created with libfoo.so.1 as a dependency, record the version names of the interfaces the dynamic object binds
to.

The following example creates an application that references symbols foo1 and foo2. The versioning dependency information that is recorded in the application can be examined using pvs(1) with the -r option.

In this example, the application prog has bound to the two interfaces SUNW_1.1 and SUNW_1.2. These interfaces provided the global
symbols foo1 and foo2 respectively.

Because version definition SUNW_1.1 is defined within libfoo.so.1 as being inherited by the version definition SUNW_1.2, you only need to record the one dependency. This inheritance provides for the normalization of version definition
dependencies. This normalization reduces the amount of version information that is maintained within an object. This normalization also reduces the version verification processing that is required
at runtime.

Because the application prog was built against the shared object's implementation containing the weak version definition SUNW_1.2.1, this dependency
is also recorded. Even though this version definition is defined to inherit the version definition SUNW_1.2, the version's weak nature precludes its normalization with SUNW_1.1. A weak version definition results in a separate dependency recording.

Had there been multiple weak version definitions that inherited from each other, then these definitions are normalized in the same manner as non-weak version definitions are.

Note –

The recording of a version dependency can be suppressed by the link-editor's -z noversion option.

The runtime linker validates the existence of any recorded version
definitions from the objects that are bound to when the application is executed. This validation can be displayed using ldd(1) with the -v option. For example, by running ldd(1) on the application prog, the version definition dependencies are shown to be found correctly
in the dependency libfoo.so.1.

ldd(1) with the -v option implies verbose output. A recursive list of all dependencies, together with all versioning requirements, is generated.

If a non-weak version
definition dependency cannot be found, a fatal error occurs during application initialization. Any weak version definition dependency that cannot be found is silently ignored. For example, if the
application prog is run in an environment in which libfoo.so.1 only contains the version definition SUNW_1.1, then the following fatal
error occurs.

If prog had not
recorded any version definition dependencies, the nonexistence of the symbol foo2 could result in a fatal relocation error a runtime. This relocation error might occur at process
initialization, or during process execution. An error condition might not occur at all if the execution path of the application did not call the function foo2. See Relocation Errors.

A version definition dependency provides an alternative and immediate indication of the availability of the interfaces required by the application.

For example, prog might run in an environment in which libfoo.so.1 only contains the version definitions SUNW_1.1 and SUNW_1.2. In this event, all non-weak version definition requirements are satisfied. The absence of the weak version definition SUNW_1.2.1 is deemed nonfatal. In this case,
no runtime error condition is generated.

At runtime, if an implementation of a dependency contains no version definition information, then any version verification of the dependency is silently ignored. This policy provides a level
of backward compatibility as a transition from non-versioned to versioned shared objects occurs. ldd(1) can always be used to display any version requirement discrepancies.

Note –

The environment variable LD_NOVERSION can be used to suppress all runtime versioning verification.

Verifying Versions in Additional Objects

Version definition symbols also provide a mechanism for verifying the version requirements of an object obtained by dlopen(3C). An object that is added to the process's address space by using dlopen(3C) receives no automatic version dependency verification. Thus, the caller
of dlopen(3C) is responsible for verifying that any
versioning requirements are met.

The presence of a required version definition can be verified by looking up
the associated version definition symbol using dlsym(3C).
The following example adds the shared object libfoo.so.1 to a process using dlopen(3C). The availability of the interface SUNW_1.2 is then verified.

The use of the dlopen(3C) flag RTLD_FIRST ensures that the dlsym(3C) search is restricted
to libfoo.so.1.

Specifying a Version Binding

When creating a dynamic object against a shared object containing version definitions, you can instruct the link-editor to limit the binding to specific
version definitions. Effectively, the link-editor enables you to control an object's binding to specific interfaces.

An object's binding requirements can be controlled using a file control directive. This directive is supplied
using the link-editor's -M option and an associated mapfile. The following syntax for file control directives is available.

name - version [ version ... ] [ $ADDVERS=version ];

name – Represents the name of the shared object dependency. This name should match the shared object's compilation environment name as used by the
link-editor. See Library Naming Conventions.

version – Represents the version definition name within the shared object that should be made available for binding. Multiple version definitions
can be specified.

$ADDVERS – Allows for additional version definitions to be recorded.

The control of version binding can be useful in the following scenarios.

When a shared object defines independent, unique versions. This versioning is possible when defining different standards interfaces. An object can be built with binding controls
to ensure the object only binds to a specific interface.

When a shared object has been versioned over several software releases. An object can be built with binding controls to restrict its binding to the interfaces that were available
in a previous software release. Thus, an object can run with an old release of the shared object dependency, after being built using the latest release of the shared object.

The following example illustrates the use of the version control mechanism. This example uses the shared object libfoo.so.1 containing the following version interface
definitions.

The version definitions SUNW_1.1 and SUNW_1.2 represent interfaces within libfoo.so.1 that were made available in software Release X and Release X+1 respectively.

An application can be built to bind only to the interfaces available in Release X by using the following version control mapfile directive.

$ cat mapfile
libfoo.so - SUNW_1.1;

For example, suppose you develop an application, prog, and want to ensure that the application can run on Release X. The application must only use the interfaces available in Release X.
If the application mistakenly references the symbol bar, then the application is not compliant with the required interface. This condition is signalled by the link-editor as an
undefined symbol error.

To be compliant with the SUNW_1.1 interface, you must remove the reference to bar. You can either rework the application to remove the requirement on bar, or add an implementation of bar to the creation of the application.

Note –

By
default, shared object dependencies encountered as part of a link-edit, are also verified against any file control directives. Use the environment variable LD_NOVERSION to suppress
the version verification of any shared object dependencies.

Binding to Additional Version Definitions

To record more version dependencies than would be produced from the normal symbol binding of an object, use the $ADDVERS file control directive. The following sections describe
scenarios where this additional binding can be useful.

Redefining an Interface

One scenario is the consumption of an ISV specific interface into a public standard interface.

From the previous libfoo.so.1 example, assume that in Release X+2, the version definition SUNW_1.1 is subdivided into two
standard releases, STAND_A and STAND_B. To preserve compatibility, the SUNW_1.1 version definition must be maintained. In this example, this
version definition is expressed as inheriting the two standard definitions.

If the only requirement of application prog is the interface symbol foo1, the application will have a single dependency on the version definition STAND_A. This precludes running prog on a system where libfoo.so.1 is less than Release X+2. The version definition STAND_A did not exist in previous releases, even though the interface foo1 did.

The application prog can be built to align its requirement with previous releases by creating a dependency on SUNW_1.1.

This explicit dependency is sufficient to encapsulate the true dependency requirements. This dependency satisfies compatibility with older releases.

Binding to a Weak Version

Creating a Weak Version Definition described
how weak version definitions can be used to mark an internal implementation change. These version definitions are well suited to indicate bug fixes and performance improvements made to an object.
If the existence of a weak version is required, an explicit dependency on this version definition can be generated. The creation of such a dependency can be important when a bug fix, or performance
improvement, is critical for the object to function correctly.

From the previous libfoo.so.1 example, assume a bug fix is incorporated as the weak version definition SUNW_1.2.1 in software Release X+3:

Normally, if an application is built against this libfoo.so.1, the application records a weak dependency on the version definition SUNW_1.2.1. This
dependency is informational only. This dependency does not cause termination of the application should the version definition not exist in the implementation of libfoo.so.1 that
is used at runtime.

The file control directive, $ADDVERS, can be used to generate an explicit dependency on a version definition. If this definition is weak, then this explicit reference also
the version definition to be promoted to a strong dependency.

The application prog can be built to enforce the requirement that the SUNW_1.2.1 interface be available at runtime by using the following file control
directive.

prog has an explicit dependency on the interface STAND_A. Because the version definition SUNW_1.2.1 is promoted to a strong version,
the version SUNW_1.2.1 is normalized with the dependency STAND_A. At runtime, if the version definition SUNW_1.2.1 cannot be found, a fatal
error is generated.

Note –

When working with a small number of dependencies, you can use the link-editor's -u option to explicitly bind to a version definition. Use this option to reference the
version definition symbol. However, a symbol reference is nonselective. When working with multiple dependencies, that contain similarly named version definitions, this technique might be insufficient
to create explicit bindings.

Version Stability

Various models have been described that provide for binding to a version definition within an object. These models allow for the runtime validation of interface requirements. This verification
only remains valid if the individual version definitions remain constant over the life time of the object.

A version definition for an object can be created for other objects to bind with. This version definition must continue to exist in subsequent releases of the object. Both the version name
and the symbols associated with the version must remain constant. To help enforce these requirements, wildcard expansion of the symbol names defined within a version definition is not supported.
The number of symbols that can match a wildcard might differ over the course of an objects evolution. This difference can lead to accidental interface instability.

Relocatable Objects

The previous sections have described how version information can be recorded within dynamic objects. Relocatable objects can maintain versioning information in a similar manner. However, subtle
differences exist regarding how this information is used.

Any version definitions supplied to the link-edit of a relocatable object are recorded in the object. These definitions follow the same format as version definitions recorded in dynamic objects.
However, by default, symbol reduction is not carried out on the relocatable object being created. Symbol reductions that are defined by the versioning information are applied to the relocatable
object when the object is used to create a dynamic object.

In addition, any version definition found in a relocatable object is propagated to the dynamic object. For an example of version processing in relocatable objects, see Reducing Symbol Scope.

Note –

Symbol reduction that is implied by a version definition can be applied to a relocatable object by using the link-editors -B reduce option.

External Versioning

Runtime references to a shared object should always refer to the versioned file name. A versioned file name is usually expressed as a file name with a version number suffix.

Should a shared object's interface changes in an incompatible manner, such a change can break old applications. In this instance, a new shared object should be distributed with a new versioned
file name. In addition, the original versioned file name must still be distributed to provide the interfaces required by the old applications.

You should provide shared objects as separate versioned file names within the runtime environment when building applications over a series of software releases. You can then guarantee that
the interface against which the applications were built is available for the application to bind during their execution.

The following section describes how to coordinate the binding of an interface between the compilation and runtime environments.

Coordination of Versioned Filenames

A link-edit commonly references shared object dependencies using the link-editors -l option. This option uses the link-editor's library search mechanism to locate shared objects
that are prefixed with lib and suffixed with .so.

However, at runtime, any shared object dependencies should exist as a versioned file name. Instead of maintaining two distinct shared objects that follow two naming conventions, create file
system links between the two file names.

For example, the shared object libfoo.so.1 can be made available to the compilation environment by using a symbolic link. The compilation file name is a symbolic link
to the runtime file name.

Either a symbolic link or hard link can be used. However, as a documentation and diagnostic aid, symbolic links are more useful.

The shared object libfoo.so.1 has been generated for the runtime environment. The symbolic link libfoo.so, has also enabled this file's use in a compilation
environment.

$ cc -o prog main.o -L. -lfoo

The link-editor processes the relocatable object main.o with the interface described by the shared object libfoo.so.1, which is found by following
the symbolic link libfoo.so.

Over a series of software releases, new versions of libfoo.so can be distributed with changed interfaces. The compilation environment can be constructed to use the interface
that is applicable by changing the symbolic link.

In this example, three major versions of the shared object are available. Two versions, libfoo.so.1 and libfoo.so.2, provide the dependencies for
existing applications. libfoo.so.3 offers the latest major release for creating and running new applications.

The use of this symbolic link mechanism solely is insufficient to coordinate the compilation shared object with a runtime versioned file name. As the example presently stands, the link-editor
records in the dynamic executable prog the file name of the shared object the link-editor processes. In this case, that file name seen by the link-editor is the compilation
environment file.

When the application prog is executed, the runtime linker searches for the dependency libfoo.so. prog binds to the file to which this
symbolic link is pointing.

To ensure the correct runtime name is recorded as a dependency, the shared object libfoo.so.1 should be built with an soname definition. This definition identifies the shared object's runtime name. This name is used as the dependency name by any object that links
against the shared object. This definition can be provided using the -h option during the creation of the shared object.

This symbolic link and the soname mechanism establish a robust coordination between the shared-object naming conventions of the compilation and runtime environment. The interface
processed during the link-edit is accurately recorded in the output file generated. This recording ensures that the intended interface are furnished at runtime.

Multiple External Versioned Files in the Same Process

The creation of a new externally versioned shared object is a major change. Be sure you understand the complete dependencies of any processes that use a member of a family of externally
versioned shared objects.

For example, an application might have a dependency on libfoo.so.1 and an externally delivered object libISV.so.1. This latter object might also have
a dependency on libfoo.so.1. The application might be redesigned to use the new interfaces in libfoo.so.2. However, the application might not change the
use of the external object libISV.so.1. Depending on the scope of visibility of the implementations of libfoo.so that get loaded at runtime, both major
versions of the file can be brought into the running process. The only reason to change the version of libfoo.so is to mark an incompatible change. Therefore, having both versions
of the object within a process can lead to incorrect symbol binding and hence undesirable interactions.

The creation of an incompatible interface change should be avoided. Only if you have full control over the interface definition, and all of the objects that reference this definition, should
an incompatible change be considered.