Friday, 15 August 2014

One of my open source projects is python-gphoto2, a Python interface to the popular libgphoto2 digital camera software library. It uses SWIG (Simplified Wrapper and Interface Generator) to generate the Python bindings from the library's "header files", saving me the trouble of hand crafting a Python interface to every C function in the library.

SWIG makes it quite easy to produce a "low level" Python interface to a C (or C++) library, but the result can be quite tricky to use. Problems often occur with object creation and deletion, which is something the average Python programmer doesn't normally have to think about. This blog post describes how I've tackled these problems in python-gphoto2.

The libgphoto2 C library makes a lot of use of "opaque" structs. These have their contents hidden from the library user, who only sees them as pointers to an unknown type. This is a classic example of good user interface design, and fits well with Python, where everything is a pointer anyway.

"Foo" objects are created by foo_new and deleted by foo_free. Data stored in the Foo object can only be accessed through functions such as foo_get_value_a and foo_set_value_b. All the functions return an int to indicate success or failure.

The first challenge when writing a SWIG interface file for the Foo object is raised by foo_new, which returns its result in a pointer to a pointer. SWIG's "typemap" system is well suited to dealing with this, as in this interface file:

The in typemap tells SWIG that any function in Foo.h that has a Foo ** parameter does not require an input value. The argout typemap is a bit more complicated. It is also applied to any function that has a Foo ** parameter. The first five lines convert the existing function result to a Python list, if it isn't already a list. The next three lines create a new Python Foo * pointer from the function's output and append it to the result list. The upshot of all this is that Foo objects can then be used from Python as follows:

As a result of passing SWIG_POINTER_NEW to SWIG_NewPointerObj we now get a "possible memory leak" warning if a Python program fails to call foo_free. The %delobject directive makes it safe to call foo_free twice. (See the SWIG documentation for more detail.) However, the Python programmer is still responsible for calling foo_free shortly before deleting the corresponding Python object.

SWIG makes it possible to add functions to a C structure with the %extend directive, which is often used to add a destructor. My first attempts to use this failed, typically with a SWIG error "%extend defined for an undeclared class _Foo". The solution turned out to be to declare an empty struct with the appropriate name: struct _Foo {};. Because the interface hides the detail of the struct we're using, it apparently doesn't matter if it has no contents at all!

This "solution" generates yet another problem. We now have a partially defined struct which SWIG wants to generate its own constructor and destructor for. (It does this for all structs by default.) The details of the struct are hidden so we get this SWIG error: "invalid application of ‘sizeof’ to incomplete type ‘struct _Foo‘". To prevent this error we can tell SWIG to ignore the struct with the %ignore directive.

At last SWIG has all the information it needs. It knows _Foo is a struct, but it knows nothing of its contents. It knows foo_new creates Foo objects and it knows foo_free deletes Foo objects. Finally it puts the pieces together and invokes foo_free automatically when the Python Foo object is deleted.