programmer things

Lazy Sequences in Common Lisp

Jul28th, 201212:00 am

0. Delayed Evaluation with Closures

Last week
I expressed some anti-lazy sentiment, or at least it was perceived as
such. Really, I am just not a fan of laziness by default. Common Lisp is
decidedly not lazy by default, but it is easy to defer or avoid
computation with closures. With macros, we can add it to the language in
an easy-to-use form.

Common Lisp is not a lazy language, but it is easy enough to add lazy
sequences. We can model them after the normal list which is constructed
piecemeal with the cons operator. The following recursive function
builds a list of numbers counting down to 0 using cons.

This is a common pattern, but there are some shortcomings. First of all,
it’s not a tail recursive function, so this call will always build the
stack since tail call optimization or tail recursion elimination does
not apply, and will do so linearly with the argument n. Also, it
constructs the entire list. You may want this list of numbers, but you
don’t need all of them right away sitting in memory. In the example
above, the call to reduce gets a full list of 101 elements, from 100 to
0 before processing. Depending on the length of the sequence, a
different algorithm might work better.

One way (of many) to construct lazy sequences in Common Lisp is to store
the CDR (second part) of the CONS cell as a thunk, or a closure that
takes no argument and computes the next CONS in the list.

We can grab the first element of the lazy list with CAR per normal, but
to get the next CONS cell, we need a version of CDR which FUNCALLs the
thunk. That will call LAZY-COUNTDOWN which will return another CONS
cell.

1. Use CLOS and Macros for a Nice Interface

In order to use this pattern easily, we will create a nice interface.
We’ll package up the lazy list into its own class, which I will call
LCONS and use a macro of the same name to construct lazy lists by
automatically wrapping the second argument in a thunk.

Notice that LMAPCAR is lazy, too. It returns an LCONS (lazy list). In
this way we can build up a collection of functions just like the normal
functions on lists in Common Lisp, but ones that take and return lazy
lists.

Just like lazy sequences in other languages, these sequences can be
infinite. Here’s a function INTEGERS which builds an infinite lazy list
of integers.

I have encoded this into a project which is available on github:
slow-jam . It is the
above code with a few extra functions that take and/or return lazy
sequences. Here are a few examples of things you can do with the
library:

One advantage of this approach is that the tail of the lazy list is
never cached which means that, unless you use the TO-LIST function
provided in the library to turn a lazy list into a normal list, all the
elements of your lazylists (except the first) can be garbage collected.
These lists never take much permanent memory. The downside is that
results are not cached, and you may end up recalculating the same list
elements more than once. For that reason, these lists should be built in
using side-effect-free functions. If you do want to cache the contents
of a lazy list, just call TO-LIST which returns real list and keep a
reference to that.

This is just one implementation of lazy sequences in Common Lisp. It is
partially inspired by Clojure, Haskell, and SICP. If you like the
slow-jam library and want
it to be better, report an issue or make a pull request at github.