Macro-writing Macros

codeclojure

… in which we explore the power of macros, and macro-writing
macros, to DRY out repetitive code.

I’ve been writing Clojure code full time for nearly two years now. I
have a pretty good feel for the language, its virtues and its
faults. Mostly, I appreciate its virtues (though I still wish the REPL
started faster).

For me one of the language’s attractions has always been that it’s a
Lisp — a “homoiconic” language, i.e., one defined in terms of its own
data structures. Homoiconicity has one primary virtue, which is that
it makes metaprogramming more powerful and straightforward than it is
in non-homoiconic languages (arguably at some cost to readability).

In Lisp, this metaprogramming is accomplished with macros, which are
functions that transform your code during a separate stage of
compilation. In other words, you write little programs to change your
programs before they execute. In effect, you extend the compiler
itself.

I run a Clojure study group at work and find that it can be hard to
explain the utility (or appeal) of this to newcomers to Lisp. This is
partly because macros do things you can’t easily do in other
languages, and because the things you want to do tend to relate to
abstractions latent in a particular codebase.

The shape of a program should reflect only the problem it needs to
solve. Any other regularity in the code is a sign, to me at least,
that I’m using abstractions that aren’t powerful enough— often that
I’m generating by hand the expansions of some macro that I need to
write1.

In Quil, there are multiple situations in which one needs to create a
temporary context to carry out a series of operations, restoring the
original state afterwards:

In this example code, the contexts with-matrix, etc. appear so often
that the resulting savings in lines of code and mental overhead for
the reader is substantial.

However, the astute reader will realize that the macro definitions
themselves are pretty repetitive—in fact, they look almost identical
except for the setup and teardown details (this kind of “context
manager” pattern is common enough that Python has its own language
construct for it).

I generally reach for macros when I have a pattern that occurs with
obvious repetition that’s not easy to abstract out using just pure
functions. Control abstractions such as loops or exception handling
are common examples. (I find this situation occurs especially
frequently when writing test code).

In any case, the solution for our repetitive macros could be something
like:

These are exactly equivalent to the three context macros (with-*)
defined above.

With a little effort, it’s actually not too hard to construct such a
nested macro. It’s largely a matter of writing out the code you want
to generate, and then writing the code that generates it, testing with
macroexpand-1 at the REPL as you go. This page by A. Malloy has a lot
of helpful remarks, including this cautionary note: “Think twice
before trying to nest macros: it’s usually the wrong answer.” In this
case, I actually think it’s the right answer, because the pattern of a
context with setup and teardown is so common that I know I’ll reuse
this macro for many other things—we have effectively added one of my
favorite Python features to Clojure in just a few lines of code2

There’s a saying in the Clojure community: data > functions > macros.
I’m a big believer in this. Clojure’s powerful built-in
abstractions for wrangling data in all its forms make it the language
I prefer above all others these days. But occasionally that means
wrangling the data that is the code itself, thereby reaping the
benefits in power, brevity and expressiveness.

Footnotes:

To
be even more like Python’s context managers, defcontext would want
to enable the user to bind some local state resulting from the setup
phase of execution (“ with x() as y: ” idiom); examples include file
descriptors or database connections. This is left as an exercise for
the reader.