1 Case study: AntVis

This case study is a graphical simulation of ant foraging that demonstrates
the use of software transactional memory. A Haskell version of
the simulation was written by Jeff Foster and described in detail
on his blog.

There were some performance and memory usage problems with the
first version of the simulation. Another
blog post
details Jeff's use of profiling to solve the problems.

2 Case study: Processing XML using HaXML

I have a script that converts from an XML format to some pickled data structures via Data.Binary. The XML part is generated by HaXml's DtdToHaskell. On a 54M XML file, the thing swaps like crazy and takes several hours. I would like to improve the situation.

2.1 Preliminaries

2.1.1 Preflight checklist

Are you using compiler flags for optimised code, e.g. -O2?

Are you using the latest version of your libraries?

If you just set the an optimisation code, did you remember to make clean (or the equivalent) and rebuild?

2.1.2 Enable profiling on libraries

For example, my script uses HaXmL, which uses a library called polyparse:

If you are looking to profile library code that a particular executable invokes, there's one more step. This presentation is specific to GHC's profiling; look to your compiler's manual if you're not using GHC.

That extra -auto-all bit tells the profiler to track the internals of that library more carefully. You might refine the tracking later (see the GHC manual's chapter on profiling), but -auto-all is usually a good place to start.

Let's summarize.

Every library the code you want to profile depends on (transitively) must be compiled with --enable-library-profiling.

Your executable must be compiled with --enable-executable-profiling.

If you want to profile the code of a library, then that library needs to be compiled with GHC's -auto-all option (or comparable for other compilers) in addition to --enable-library-profiling. (Cabal might have a generic flag for this after version 1.8, according to this ticket.)

This article proceeds under the assumption that you are profiling an executable's code, but it's the same basic idea if you're investigating a library's code.

2.1.4 Get toy data

My script takes hours to convert 50M of XML. Running it on such data every time I tweak something would clearly not be a good idea. You want something which is small enough for your program to come back relatively quickly, but large enough to study.

I use something like sed -f makeToy.sed reallyBigFile.xml > toy.xml where makeToy.sed is a bit of text-hacking to chop off the rest of my data after the arbitrarily chosen item #6621:

/6621/{
c\
</grammar>
q
}

2.2 Test harness

Make things easy on yourself! I find that it's very helpful to automate my way out of my clumsiness. Ideally, each tweak you make to your software should be accompanied by a simple run and not some long sequence of actions, half of which you might forget. Note: you might also consider using a Makefile instead of a bunch of scripts.

We'll be working with a stable and unstable repository. It's possible that you'll be making a lot of small modifications to your program, so what would be nice is to be able to save some of your modifications along the way. Darcs is very handy for this.