ffipkg: a neat bindings creation tool. Usage example.

The ﻿FFI Packaging Utility (ffipkg) is a surprisingly easy tool for creating bindings from C to Haskell. It works by processing header files and creating a bunch of modules with accessors to structures, bindings for calling functions and all the usual boilerplate you usually need to write when doing FFI. What it doesn’t do is to provide a nice, high level binding to the underlying C library. But with the boring part worked out we can do it ourselves in Haskell.

In this post I’ll show how to create a working binding to CURAND library. For those unfamiliar with CUDA: CURAND is a library for creating a high-quality pseudo random numbers directly on the graphic card’s GPU. It works with any recent NVidia card.

First let’s install ffipkg itself. Actually the package name is different – hsffig. What you need to do is:

$ cabal install hsffig

There shouldn’t be any problems with that.

The next step would be installing the CUDA toolkit. If you are on Linux and your distribution provides it prepackaged you can install it with your package manager. Otherwise proceed to NVidia site and download the right installer.

I used the one for Ubuntu and it worked very well. After running it gently asked for installation prefix and it simply unpacked into it. It also provides you with information on how to configure your system to make libraries and headers visible to the compilers.

Creating the binding library is dead simple. Just create and cd into directory, like this:

$ mkdir hs-curand && cd hs-curand

Then call ffipkg:

$ ffipkg -I${CUDA_INCLUDE_PATH} -L${CUDA_LINK_PATH} -lcurand curand.h

If the CUDA toolkit is installed system-wide you can omit the extra -I/-L options. Otherwise you should point ffipkg to the right directories, just like you would do with the C/C++ compiler. As an example, I put my installation into ~/localopt/cuda folder and my machine is 64 bit, so in my case I called: (Beware of ~ as it may not be expanded into $HOME)

Makefile – not to be called directly. (ffipkg will silently overwrite whatever Makefile is already in the directory)

A plenty of .hs files for each C datatype declared

Single .hsc file with wrapping code

As we examine the files a fun fact becomes apparent: not only did ffipkg provide bindings to CURAND. The bindings to the rest of CUDA are present as well! With a simple command we have created bindings to entire or most of CUDA runtime and driver API along with the CURAND library.

The next step is an actual installation which should go fine:

$ cabal install

We have installed a library, but will it actually work? Let’s find out by writing an example program.

The CURAND documentation itself contains a small program for host API which we will use:

The code is mostly 1-to-1 mapping from C code, as it should be: both programs use the same API functions. The only nuisance in Haskell is the need for extra malloc/free pair whenever API function needs a pointer to variable. This part could (and should!) be done using alloca function in the real world code, but I’d rather have more understandable version of code here.

After compiling and running this program we will see almost the same output as from the C version. The only difference is caused by different behavior of printf for floats in Haskell and in C. The actual numbers are the same under the hood.

So ffipkg has actually created a working Haskell bindings for CUDA and CURAND without any sweat. Like I said already, it’s pretty low-level – much like the *-bindings packages on Hackage or even more. This can however serve as a building block for much more Haskell-like API afterwards.

It’s not always this good. There are a few downsides of ffipkg:

It is hard to create a library ready for upload to Hackage

Requires GNU make or equivalent (Makefiles)

Documentation is disabled by default. Even if we enable it in Setup.hs by hand it still doesn’t provide a lot of useful information.

Exploring the created library hard. My best find: ghci with :+m HS_library-name_H

It fails for many packages with unhelpful error message

Sometimes it requires post-processing of created package by hand to make it work

It contains a LEXER package (a flex-based tokenizer written in C) which gets build (though not rebuild when ghc was upgraded), and creates a “transient” package not remembered by Hackage.

2. It does. It also does split-objs when building the package archive. Once cabal-install can do split-objs perhaps there will be no need in such contrived process as it is now. Besides, how much can be done without make these days? You’d need it to compile your foreign library anyway, correct?

3. I think if haddock is run on the generated Haskell, function index and signatures will be produced. I am not sure how could I transfer comments from C header (if there are any) unless there is any standard. I believe that comments get removed by the preprocessor anyway.

4. Agreed. However the foreign library has its own docs. It was my belief that having almost one-to-one language mapping makes using the library docs easier. This was the case for me with Freetype and OpenGL.

5. Unfortunately it does if there are weird things like named members of an anonymous structure member of a larger structure (makes sense?). This in fact was an actual bug that was fixed some time ago upon someone’s comment. Header files of libc sometimes introduce things not so much anticipated 😉

@1.: I believe Gtk2hs has had similar problems before it could finally be cabalized. Still it depends on gtk2hs-buildtools, so they didn’t solve the problem completely. As to webidl way of building – it’s not that portable. It didn’t build on GHC 6.12 and 7.0. The error message was:
> cabal: cannot configure webidl-0.1.1. It requires LEXER -any
> There is no available version of LEXER that satisfies -any

@2.: I didn’t see it’s the split-objs feature that require ‘make’. Anyway, on Windows installing GNU make is a bit of a hassle, which most people won’t go into. The library might come in already compiled form, which is mostly true even for open source projects. On most Linux distributions for example you will find packages with headers and libraries, but rarely with source code. On Linux however installing GNU make is totally painless.

@3.: I don’t know of any generic way to do this. Perhaps the best you can do is actually quote the header part that was used to generate the binding.

@5.: The C language is deceivingly simple at first look, but the great number of nonstandard extensions and weird rules can bite. The great example is language-c package on Hackage: http://hackage.haskell.org/package/language-c

@6.: I cannot give an example right now, but in generally the resulting hs/hsc code was incorrect and needed some hand fixes before it could be compiled into Haskell lib.

@1: this was exactly what I meant: LEXER was not remembered by Hackage (being intentionally transient) and was not rebuilt automatically (because cabal clean or whatever similar was likely not run). Try downloading the package and building it fresh: it will build at least with 6.12 (not tested with 7.01). One more thing to mention: the webidl approach only works with self-contained foreign code. OTOH library-based bindings are always site-specific (per concrete library installation paths), thus it makes no sense to store them on Hackage. I’d recommend to generate them privately per-project and use capri or cabal-dev.

When hsffig was first developed (2005), Cabal was in infancy, and Hackage did not even exist. Ffipkg was the reponse to growing Cabal popularity. Having permanent packages with auto-generated bindings on Hackage is a bad idea: Hackage must have all these foreign libraries installed in order for such packages to show as buildable. Plus, the toplevel namespace will be polluted.

@2 split-objs is essential: without it generated binary would be much bigger as the linker does not always optimize the build properly. I never tried ffipkg on windows (nor did I do any development on windows at all). I am not a big fan of distros either: I mostly use them for base install; then compiling everything I need from tarballs.

@5 I’m afraid the C language parser for hsffig/ffipkg was written before the language-c package became available. Otherwise I wouldn’t spend time writing the parser myself.

@6 although hsffig/ffipkg has been around since 2005, there have been just a handful of users I heard from. The more people use it, the more feedback is generated, the more bugs are fixed. Any specific examples would be of help.