SBCL 1.0.5.28 includes an experimental code coverage tool (sb-cover) as a new
contrib module. Basically you just need to compile your code with a special
optimize proclamation, load it, run some tests, and then run a reporting
utility. The reporting utility will produce some html files. One will contain an
aggregate coverage report of your whole system, the others will show your
source code transformed into angry fruit salad:

For a more substantial example, here's
the coverage output for the cl-ppcre test suite.

There are still some places where the coverage output won't be what
most people would intuitively expect. Some, like the handling of inlined
functions, would be simple to solve. It's just not yet clear to me what
the right solution would be.
For example in the case of inlined functions the right solution might be
suppressing inlining when compiling with coverage instrumentation, or it might
be to say "don't do that, then" to the users. Others are fundamentally
unsolvable, due to the impossibility of reliably mapping the forms that
the compiler sees back to the exact character position in the source file.
Hopefully this'll still turn out to be useful in its current state.

Comments

By Ondrej Svitek on 2007-05-03

That's a neat contrib!

What about adding support for measuring hotness of code? Hotspots would naturally emerge as islands of bright red in the sea of cold blue (or similar suggestive color mapping). It could give one a 'feel' of program execution, as we humans excel at decoding and comprehending visual information.

Of course, it wouldn't be meant as a replacement for real code profiling. Just a fancy tool for quick visual identification of code portions under heavy load, which require further attention.

I believe you have all the necessary bits and pieces to make something like this possible.

Anyway, thanks for the great work, keep it up!

By Juho on 2007-05-03

A couple of years ago I once wrote an sb-sprof extension that could annotate the original source code forms with the corresponding amount of profiler ticks:

I think the lesser languages benefit a lot more from this kind of tool than lisp but it's good to know that we have it if we need it.

Thanks!

By Thom on 2007-05-10

anon,

As someone who spends a lot of time in a large, organically-grown Lisp code base, I think this might actually be as or more useful to me than the code coverage I used to run on Java systems. Of course, it was my request (among others) that indirectly led Juho to do this, I think, so I've obviously got my bias.

By Paul Dietz on 2007-06-25

This is really cool, thanks.

I've been fiddling with Waters' old COVER package, which doesn't have the spiffy HTML interface. Not sure how hard that would be to add. You might want to look at how it handles annotating/not annotating code, particularly in macro expansions.

One thing I've wanted to do with COVER is make it more extensible, so, for example, it could be used with an automated test generator to evolve tests that cover untested parts of the code. It would be nice to make SB-COVER user extensible too.