= Using Clozure CL's FFI =
== BLAS and LAPACK libraries ==
The BLAS and LAPACK libraries were originally written in Fortran.
It's certainly possible to call the Fortran-style routines directly
from CCL, but some care is needed, because Fortran conventions differ
from C conventions. (CCL's FFI is geared towards calling C
functions.)
In Fortran, all arguments to a function, even scalar values, are passed
by reference (that is, as pointers). Arrays are also different. In
Common Lisp, arrays are indexed from 0 and are stored in row-major
order. In Fortran, arrays are indexed from 1, and are stored in
column-major order.
To show how this works in practice, we'll call the BLAS routine
idamax, which finds the index of the element of a double-float vector
with the maximum absolute value.
{{{
Welcome to Clozure Common Lisp Version 1.7-dev-r14780 (LinuxARM32)!
? (open-shared-library "libblas.so")
#
? (rletz ((n :int 10)
(dx (:array :double 10))
(incx :int 1))
(setf (paref dx (:array :double) 5) 100d0
(paref dx (:array :double) 8) -1000d0)
(external-call "idamax_" (:* int) n (:* double) dx (:* int) incx :int))
9
?
}}}
Notice that the function parameters are passed by reference (that is,
as pointers). Second, note that the returned answer is 9 and not 8 as
we might expect. This is because Fortran array indexes start at 1,
not at 0 as they do in Common Lisp. Finally, note the "_" suffix on
the name of the Fortran function---that's simply a common Unix
convention.
There is also a C interface to the BLAS. This is somewhat more
convenient to use, as the following example shows:
{{{
? (rletz ((x (:array :double 10)))
(setf (paref x (:array :double) 5) 100d0
(paref x (:array :double) 8) -1000d0)
(external-call "cblas_idamax" :int 10 (:* double) x :int 1 :int))
8
}}}
With the C interface, we don't have to pass scalar parameters by reference,
and the indexes are 0-based, as a Lisp programmer would expect. (The
libblas.so that we loaded earlier also contains the C interface functions
on this system.)
We can make this more convenient still by using the interface
translator to parse the cblas.h header file and generate an interface
database for it.
The directions at http://trac.clozure.com/ccl/wiki/CustomFramework
explain how to do this. With the interface database present, calling
the BLAS function becomes even easier:
{{{
? (use-interface-dir :cblas)
#
? (rletz ((x (:array :double 10)))
(setf (paref x (:array :double) 5) 100d0
(paref x (:array :double) 8) -1000d0)
(#_cblas_idamax 10 x 1))
8
}}}
Now we can use the handy #_ reader macro, and we no longer have to
provide the return type of the function, or the types of its
arguments. The #_ reader macro looks them up in the interface
database and generates the correct external-call form automatically.
== Lisp Arrays and Foreign Arrays ==
You may be wondering if there is some way to pass lisp arrays to
foreign functions. In CCL, this is generally not possible. Even if
it were possible to obtain a pointer to the array data, and assuming
that the lisp array data is stored in the format that the foreign
function expects, the GC might run at any time and move the underlying
object. (Keep in mind that CCL uses multiple threads, and any thread
might initiate a GC at arbitrary times.)
Data in a lisp array must therefore be copied to memory either on the
stack or on the foreign heap before being passed to a foreign
function.
There are, however, a few features in CCL that may be useful for
avoiding the need to copy.
The first is an object called a heap ivector.
http://ccl.clozure.com/ccl-documentation.html#f_make-heap-ivector
Here's an example. (Presumably you would keep the heap ivector object
around somewhere, and free it when you were completely done with it.)
{{{
(multiple-value-bind (vector ptr)
(ccl:make-heap-ivector 10 'double-float)
(setf (aref vector 5) 100d0
(aref vector 8) -1000d0)
(prog1
(#_cblas_idamax 10 ptr 1)
(ccl:dispose-heap-ivector vector)))
}}}
The second is a macro called ccl:with-pointer-to-ivector. See the
docstring. This must be used with some care, because it inhibits
the GC within its body.
{{{
(let ((vector (make-array 10 :element-type 'double-float)))
(setf (aref vector 5) 100d0
(aref vector 8) -1000d0)
(ccl:with-pointer-to-ivector (ptr vector)
(#_cblas_idamax 10 ptr 1)))
}}}
The nice thing about the above two examples is that you can still
access the vector in the normal way.
In some earlier examples, we used ccl:rlet to allocate a vector on the
stack. We can also allocate memory on the foreign heap either with
#_malloc or with the slightly higher-level ccl:make-record. In either
case, you must explicitly call ccl:free.
{{{
(let ((x (ccl:make-record (:array :double 10))))
(setf (ccl:paref x (:array :double) 5) 100d0
(ccl:paref x (:array :double) 8) -1000d0)
(prog1
(#_cblas_idamax 10 x 1)
(ccl:free x)))
}}}
It's also possible to use ccl:make-gcable-record to allocate foreign
memory that will be freed when it is unreachable from lisp.
{{{
(let ((x (ccl:make-gcable-record (:array :double 10))))
(setf (paref x (:array :double) 5) 100d0
(paref x (:array :double) 8) -1000d0)
(#_cblas_idamax 10 x 1))
}}}
This works well as long as the gcable pointer is the only thing
(either in the lisp world or foreign world) that refers to the memory.
See the manual for a more detailed explanation:
http://ccl.clozure.com/ccl-documentation.html#f_make-gcable-record