1.12 Providing a C API for an Extension Module

Many extension modules just provide new functions and types to be
used from Python, but sometimes the code in an extension module can
be useful for other extension modules. For example, an extension
module could implement a type ``collection'' which works like lists
without order. Just like the standard Python list type has a C API
which permits extension modules to create and manipulate lists, this
new collection type should have a set of C functions for direct
manipulation from other extension modules.

At first sight this seems easy: just write the functions (without
declaring them static, of course), provide an appropriate
header file, and document the C API. And in fact this would work if
all extension modules were always linked statically with the Python
interpreter. When modules are used as shared libraries, however, the
symbols defined in one module may not be visible to another module.
The details of visibility depend on the operating system; some systems
use one global namespace for the Python interpreter and all extension
modules (Windows, for example), whereas others require an explicit
list of imported symbols at module link time (AIX is one example), or
offer a choice of different strategies (most Unices). And even if
symbols are globally visible, the module whose functions one wishes to
call might not have been loaded yet!

Portability therefore requires not to make any assumptions about
symbol visibility. This means that all symbols in extension modules
should be declared static, except for the module's
initialization function, in order to avoid name clashes with other
extension modules (as discussed in section 1.4). And it
means that symbols that should be accessible from other
extension modules must be exported in a different way.

Python provides a special mechanism to pass C-level information
(pointers) from one extension module to another one: CObjects.
A CObject is a Python data type which stores a pointer (void
*). CObjects can only be created and accessed via their C API, but
they can be passed around like any other Python object. In particular,
they can be assigned to a name in an extension module's namespace.
Other extension modules can then import this module, retrieve the
value of this name, and then retrieve the pointer from the CObject.

There are many ways in which CObjects can be used to export the C API
of an extension module. Each name could get its own CObject, or all C
API pointers could be stored in an array whose address is published in
a CObject. And the various tasks of storing and retrieving the pointers
can be distributed in different ways between the module providing the
code and the client modules.

The following example demonstrates an approach that puts most of the
burden on the writer of the exporting module, which is appropriate
for commonly used library modules. It stores all C API pointers
(just one in the example!) in an array of void pointers which
becomes the value of a CObject. The header file corresponding to
the module provides a macro that takes care of importing the module
and retrieving its C API pointers; client modules only have to call
this macro before accessing the C API.

The exporting module is a modification of the spam module from
section 1.1. The function spam.system()
does not call the C library function system() directly,
but a function PySpam_System(), which would of course do
something more complicated in reality (such as adding ``spam'' to
every command). This function PySpam_System() is also
exported to other extension modules.

The function PySpam_System() is a plain C function,
declared static like everything else:

The #define is used to tell the header file that it is being
included in the exporting module, not a client module. Finally,
the module's initialization function must take care of initializing
the C API pointer array:

The main disadvantage of this approach is that the file
spammodule.h is rather complicated. However, the
basic structure is the same for each function that is
exported, so it has to be learned only once.

Finally it should be mentioned that CObjects offer additional
functionality, which is especially useful for memory allocation and
deallocation of the pointer stored in a CObject. The details
are described in the Python/C API
Reference Manual in the section
``CObjects'' and in the implementation
of CObjects (files Include/cobject.h and
Objects/cobject.c in the Python source code distribution).