Navigation

As well as creating normal user-defined classes with the Python class
statement, Cython also lets you create new built-in Python types, known as
extension types. You define an extension type using the cdef class
statement. Here’s an example:

As you can see, a Cython extension type definition looks a lot like a Python
class definition. Within it, you use the def statement to define methods that
can be called from Python code. You can even define many of the special
methods such as __init__() as you would in Python.

The main difference is that you can use the cdef statement to define
attributes. The attributes may be Python objects (either generic or of a
particular extension type), or they may be of any C data type. So you can use
extension types to wrap arbitrary C data structures and provide a Python-like
interface to them.

Attributes of an extension type are stored directly in the object’s C struct.
The set of attributes is fixed at compile time; you can’t add attributes to an
extension type instance at run time simply by assigning to them, as you could
with a Python class instance. However, you can explicitly enable support
for dynamically assigned attributes, or subclass the extension type with a normal
Python class, which then supports arbitrary attribute assignments.
See Dynamic Attributes.

There are two ways that attributes of an extension type can be accessed: by
Python attribute lookup, or by direct access to the C struct from Cython code.
Python code is only able to access attributes of an extension type by the
first method, but Cython code can use either method.

By default, extension type attributes are only accessible by direct access,
not Python access, which means that they are not accessible from Python code.
To make them accessible from Python code, you need to declare them as
public or readonly. For example:

cdefclassShrubbery:cdefpublicintwidth,heightcdefreadonlyfloatdepth

makes the width and height attributes readable and writable from Python code,
and the depth attribute readable but not writable.

Note

You can only expose simple C types, such as ints, floats, and
strings, for Python access. You can also expose Python-valued attributes.

Note

Also the public and readonly options apply only to
Python access, not direct access. All the attributes of an extension type
are always readable and writable by C-level access.

It is not possible to add attributes to an extension type at runtime by default.
You have two ways of avoiding this limitation, both add an overhead when
a method is called from Python code. Especially when calling cpdef methods.

The first approach is to create a Python subclass.:

cdefclassAnimal:cdefintnumber_of_legsdef__cinit__(self,intnumber_of_legs):self.number_of_legs=number_of_legsclassExtendableAnimal(Animal):# Note that we use class, not cdef classpassdog=ExtendableAnimal(4)dog.has_tail=True

Declaring a __dict__ attribute is the second way of enabling dynamic attributes.:

Before you can directly access the attributes of an extension type, the Cython
compiler must know that you have an instance of that type, and not just a
generic Python object. It knows this already in the case of the self
parameter of the methods of that type, but in other cases you will have to use
a type declaration.

because the sh parameter hasn’t been given a type, the width attribute
will be accessed by a Python attribute lookup. If the attribute has been
declared public or readonly then this will work, but it
will be very inefficient. If the attribute is private, it will not work at all
– the code will compile, but an attribute error will be raised at run time.

We here cimport the class Shrubbery, and this is necessary
to declare the type at compile time. To be able to cimport an extension type,
we split the class definition into two parts, one in a definition file and
the other in the corresponding implementation file. You should read
Sharing Extension Types to learn to do that.

Suppose I have a method quest() which returns an object of type Shrubbery.
To access it’s width I could write:

cdefShrubberysh=quest()print(sh.width)

which requires the use of a local variable and performs a type test on assignment.
If you know the return value of quest() will be of type Shrubbery
you can use a cast to write:

print((<Shrubbery>quest()).width)

This may be dangerous if quest() is not actually a Shrubbery, as it
will try to access width as a C struct member which may not exist. At the C level,
rather than raising an AttributeError, either an nonsensical result will be
returned (interpreting whatever data is at that address as an int) or a segfault
may result from trying to access invalid memory. Instead, one can write:

print((<Shrubbery?>quest()).width)

which performs a type check (possibly raising a TypeError) before making the
cast and allowing the code to proceed.

To explicitly test the type of an object, use the isinstance() builtin function.
For known builtin or extension types, Cython translates these into a
fast and safe type check that ignores changes to
the object’s __class__ attribute etc., so that after a successful
isinstance() test, code can rely on the expected C structure of the
extension type and its cdef attributes and methods.

When you declare a parameter or C variable as being of an extension type,
Cython will allow it to take on the value None as well as values of its
declared type. This is analogous to the way a C pointer can take on the value
NULL, and you need to exercise the same caution because of it. There is no
problem as long as you are performing Python operations on it, because full
dynamic type checking will be applied. However, when you access C attributes
of an extension type (as in the widen_shrubbery function above), it’s up to
you to make sure the reference you’re using is not None – in the
interests of efficiency, Cython does not check this.

You need to be particularly careful when exposing Python functions which take
extension types as arguments. If we wanted to make widen_shrubbery() a
Python function, for example, if we simply wrote:

Now the function will automatically check that sh is notNone along
with checking that it has the right type.

Note

notNone clause can only be used in Python functions (defined with
def) and not C functions (defined with cdef). If
you need to check whether a parameter to a C function is None, you will
need to do it yourself.

Note

Some more things:

The self parameter of a method of an extension type is guaranteed never to
be None.

When comparing a value with None, keep in mind that, if x is a Python
object, xisNone and xisnotNone are very efficient because they
translate directly to C pointer comparisons, whereas x==None and
x!=None, or simply using x as a boolean value (as in ifx:...)
will invoke Python operations and therefore be much slower.

Although the principles are similar, there are substantial differences between
many of the __xxx__() special methods of extension types and their Python
counterparts. There is a separate page devoted to this subject, and you should
read it carefully before attempting to use any special methods in your
extension types.

You can declare properties in an extension class using the same syntax as in ordinary Python code:

cdefclassSpam:@propertydefcheese(self):# This is called when the property is read....@cheese.setterdefcheese(self,value):# This is called when the property is written....@cheese.deleterdefcheese(self):# This is called when the property is deleted.

There is also a special (deprecated) legacy syntax for defining properties in an extension class:

cdefclassSpam:propertycheese:"A doc string can go here."def__get__(self):# This is called when the property is read....def__set__(self,value):# This is called when the property is written....def__del__(self):# This is called when the property is deleted.

The __get__(), __set__() and __del__() methods are all
optional; if they are omitted, an exception will be raised when the
corresponding operation is attempted.

Here’s a complete example. It defines a property which adds to a list each
time it is written to, returns the list when it is read, and empties the list
when it is deleted.:

An extension type may inherit from a built-in type or another extension type:

cdefclassParrot:...cdefclassNorwegian(Parrot):...

A complete definition of the base type must be available to Cython, so if the
base type is a built-in type, it must have been previously declared as an
extern extension type. If the base type is defined in another Cython module, it
must either be declared as an extern extension type or imported using the
cimport statement.

An extension type can only have one base class (no multiple inheritance).

Cython extension types can also be subclassed in Python. A Python class can
inherit from multiple extension types provided that the usual Python rules for
multiple inheritance are followed (i.e. the C layouts of all the base classes
must be compatible).

There is a way to prevent extension types from
being subtyped in Python. This is done via the final directive,
usually set on an extension type using a decorator:

cimportcython@cython.finalcdefclassParrot:defdone(self):pass

Trying to create a Python subclass from this type will raise a
TypeError at runtime. Cython will also prevent subtyping a
final type inside of the same module, i.e. creating an extension type
that uses a final type as its base type will fail at compile time.
Note, however, that this restriction does not currently propagate to
other extension modules, so even final extension types can still be
subtyped at the C level by foreign code.

Extension types can have C methods as well as Python methods. Like C
functions, C methods are declared using cdef or cpdef instead of
def. C methods are “virtual”, and may be overridden in derived
extension types. In addition, cpdef methods can even be overridden by python
methods when called as C method. This adds a little to their calling overhead
compared to a cdef method:

Cython provides two ways to speed up the instantiation of extension types.
The first one is a direct call to the __new__() special static method,
as known from Python. For an extension type Penguin, you could use
the following code:

Note that the path through __new__() will not call the type’s
__init__() method (again, as known from Python). Thus, in the example
above, the first instantiation will print eating!, but the second will
not. This is only one of the reasons why the __cinit__() method is
safer and preferable over the normal __init__() method for extension
types.

The second performance improvement applies to types that are often created
and deleted in a row, so that they can benefit from a freelist. Cython
provides the decorator @cython.freelist(N) for this, which creates a
statically sized freelist of N instances for a given type. Example:

cimportcython@cython.freelist(8)cdefclassPenguin:cdefobjectfooddef__cinit__(self,food):self.food=foodpenguin=Penguin('fish 1')penguin=Nonepenguin=Penguin('fish 2')# does not need to allocate memory!

To then create a WrapperClass object from an existing my_c_struct
pointer, WrapperClass.from_ptr(ptr) can be used in Cython code. To allocate
a new structure and wrap it at the same time, WrapperClass.new_struct can be
used instead.

It is possible to create multiple Python objects all from the same pointer
which point to the same in-memory data, if that is wanted, though care must be
taken when de-allocating as can be seen above.
Additionally, the ptr_owner flag can be used to control which
WrapperClass object owns the pointer and is responsible for de-allocation -
this is set to False by default in the example and can be enabled by calling
from_ptr(ptr,owner=True).

The GIL must not be released in __dealloc__ either, or another lock used
if it is, in such cases or race conditions can occur with multiple
de-allocations.

Being a part of the object constructor, the __cinit__ method has a Python
signature, which makes it unable to accept a my_c_struct pointer as an
argument.

Attempts to use pointers in a Python signature will result in errors like:

Cannotconvert'my_c_struct *'toPythonobject

This is because Cython cannot automatically convert a pointer to a Python
object, unlike with native types like int.

Note that for native types, Cython will copy the value and create a new Python
object while in the above case, data is not copied and deallocating memory is
a responsibility of the extension class.

By default each extension type will support the cyclic garbage collector of
CPython. If any Python objects can be referenced, Cython will automatically
generate the tp_traverse and tp_clear slots. This is usually what you
want.

There is at least one reason why this might not be what you want: If you need
to cleanup some external resources in the __dealloc__ special function and
your object happened to be in a reference cycle, the garbage collector may
have triggered a call to tp_clear to drop references. This is the way that
reference cycles are broken so that the garbage can actually be reclaimed.

In that case any object references have vanished by the time when
__dealloc__ is called. Now your cleanup code lost access to the objects it
has to clean up. In that case you can disable the cycle breaker tp_clear
by using the no_gc_clear decorator

This example tries to close a cursor via a database connection when the Python
object is destroyed. The DBConnection object is kept alive by the reference
from DBCursor. But if a cursor happens to be in a reference cycle, the
garbage collector may effectively “steal” the database connection reference,
which makes it impossible to clean up the cursor.

Using the no_gc_clear decorator this can not happen anymore because the
references of a cursor object will not be cleared anymore.

In rare cases, extension types can be guaranteed not to participate in cycles,
but the compiler won’t be able to prove this. This would be the case if
the class can never reference itself, even indirectly.
In that case, you can manually disable cycle collection by using the
no_gc decorator, but beware that doing so when in fact the extension type
can participate in cycles could cause memory leaks

@cython.no_gccdefclassUserInfo:cdefstrnamecdeftupleaddresses

If you can be sure addresses will contain only references to strings,
the above would be safe, and it may yield a significant speedup, depending on
your usage pattern.

By default, Cython will generate a __reduce__() method to allow pickling
an extension type if and only if each of its members are convertible to Python
and it has no __cinit__ method.
To require this behavior (i.e. throw an error at compile time if a class
cannot be pickled) decorate the class with @cython.auto_pickle(True).
One can also annotate with @cython.auto_pickle(False) to get the old
behavior of not generating a __reduce__ method in any case.

Manually implementing a __reduce__ or __reduce_ex__` method will also
disable this auto-generation and can be used to support pickling of more
complicated types.

Extension types can be declared extern or public. An extern extension type
declaration makes an extension type defined in external C code available to a
Cython module. A public extension type declaration makes an extension type
defined in a Cython module available to external C code.

An extern extension type allows you to gain access to the internals of Python
objects defined in the Python core or in a non-Cython extension module.

Note

In previous versions of Pyrex, extern extension types were also used to
reference extension types defined in another Pyrex module. While you can still
do that, Cython provides a better mechanism for this. See
Sharing Declarations Between Cython Modules.

Here is an example which will let you get at the C-level members of the
built-in complex object.:

from__future__importprint_functioncdefexternfrom"complexobject.h":structPy_complex:doublerealdoubleimagctypedefclass__builtin__.complex[objectPyComplexObject]:cdefPy_complexcval# A function which uses the above typedefspam(complexc):print("Real:",c.cval.real)print("Imag:",c.cval.imag)

Note

Some important things:

In this example, ctypedef class has been used. This is
because, in the Python header files, the PyComplexObject struct is
declared with:

typedefstruct{...}PyComplexObject;

At runtime, a check will be performed when importing the Cython
c-extension module that __builtin__.complex’s tp_basicsize
matches sizeof(`PyComplexObject). This check can fail if the Cython
c-extension module was compiled with one version of the
complexobject.h header but imported into a Python with a changed
header. This check can be tweaked by using check_size in the name
specification clause.

As well as the name of the extension type, the module in which its type
object can be found is also specified. See the implicit importing section
below.

When declaring an external extension type, you don’t declare any
methods. Declaration of methods is not required in order to call them,
because the calls are Python method calls. Also, as with
struct and union, if your extension class
declaration is inside a cdef extern from block, you only need to
declare those C members which you wish to access.

The part of the class declaration in square brackets is a special feature only
available for extern or public extension types. The full form of this clause
is:

[objectobject_struct_name,typetype_object_name,check_sizecs_option]

Where:

object_struct_name is the name to assume for the type’s C struct.

type_object_name is the name to assume for the type’s statically
declared type object.

cs_option is warn (the default), error, or ignore and is only
used for external extension types. If error, the sizeof(object_struct)
that was found at compile time must match the type’s runtime tp_basicsize
exactly, otherwise the module import will fail with an error. If warn
or ignore, the object_struct is allowed to be smaller than the type’s
tp_basicsize, which indicates the runtime type may be part of an updated
module, and that the external module’s developers extended the object in a
backward-compatible fashion (only adding new fields to the end of the object).
If warn, a warning will be emitted in this case.

The clauses can be written in any order.

If the extension type declaration is inside a cdef extern from
block, the object clause is required, because Cython must be able to generate
code that is compatible with the declarations in the header file. Otherwise,
for extern extension types, the object clause is optional.

For public extension types, the object and type clauses are both required,
because Cython must be able to generate code that is compatible with external C
code.

Sometimes the type’s C struct as specified in object_struct_name may use
different labels for the fields than those in the PyTypeObject. This can
easily happen in hand-coded C extensions where the PyTypeObject_Foo has a
getter method, but the name does not match the name in the PyFooObject. In
NumPy, for instance, python-level dtype.itemsize is a getter for the C
struct field elsize. Cython supports aliasing field names so that one can
write dtype.itemsize in Cython code which will be compiled into direct
access of the C struct field, without going through a C-API equivalent of
dtype.__getattr__('itemsize').

Note that the struct uses f0, f1, f2 but they are field0,
field1, and field2 in Foo. We are given this situation, including
a header file with that struct, and we wish to write a function to sum the
values. If we write an extension module wrapper:

and now Cython will replace the slow __getattr__ with direct C access to
the FooStructNominal fields. This is useful when directly processing Python
code. No changes to Python need be made to achieve significant speedups, even
though the field names in Python and C are different. Of course, one should
make sure the fields are equivalent.

Inside a Cython module, the name of an extension type serves two distinct
purposes. When used in an expression, it refers to a module-level global
variable holding the type’s constructor (i.e. its type-object). However, it
can also be used as a C type name to declare variables, arguments and return
values of that type.

When you declare:

cdefexternclassMyModule.Spam:...

the name Spam serves both these roles. There may be other names by which you
can refer to the constructor, but only Spam can be used as a type name. For
example, if you were to explicitly import MyModule, you could use
MyModule.Spam() to create a Spam instance, but you wouldn’t be able to use
MyModule.Spam as a type name.

When an as clause is used, the name specified in the as clause also takes over
both roles. So if you declare:

cdefexternclassMyModule.SpamasYummy:...

then Yummy becomes both the type name and a name for the constructor. Again,
there are other ways that you could get hold of the constructor, but only
Yummy is usable as a type name.

An extension type can be declared public, in which case a .h file is
generated containing declarations for its object struct and type object. By
including the .h file in external C code that you write, that code can
access the attributes of the extension type.