Monthly Archives: April 2011

CSS-Selectors and TALCL are both, at heart, translators from some language (css-selectors and xml respectively) into compiled common lisp functions. We translate these expressions by translating the source language into a tree of common-lisp, inserting that common-lisp into a lambda form, then calling compile on that lambda. Because this is a central part of the utility they provide, slow compilation speed can be somewhat annoying. This led to me implementing compiler macros and other caching schemes so that compiling could be handled at compile time (when possible/constantp), where the slow compilation would be less problematic.

Below are the results comparing the two compilers. My experiment confirmed Xachs findings from 2007. The lambda tree version of the compiler is MUCH quicker (~1000x in SBCL) with comparable execution speed. This hasn’t sped up the (cl-yacc produced) parser, so I think that much of the caching and compiler macro code is still valid and still saving user time, just less crucially so now. The only downside I saw was that debugging was a touch harder because there was no human readable version of the fully translated expression (which I could get from the old translator).

Earlier this week I released a couple common lisp libraries. Thus for the rest of the week I have tried to make them usable by anyone who is not me. The situation is complicated by a few facts about our development environment. Our shared lisp library directory has been accreted over more than 5 years, and some of these libraries having been patched (some in darcs and some in git and most not patched upstream). We also make heavy use of precompiled cores to speed up start up time. All of this leads to an environment that is very hard to replicate. Quicklisp to the rescue! Quicklisp makes it easy to have a lisp environment with the same set of libraries available as everybody else, which is a tremendous win when compared to our difficult-to-replicate system.

To fix the bugs other people would see in the software that builds and works fine for me, I wrote an sbcl script to perform a clean build and fetch all unknown dependencies from quicklisp. (This script is not common lisp; there is sb-xxx all over the place.) I learned quite a bit about all of the related systems, so the end result seems somewhat trivial, but perhaps it will provide decent examples. Quicklisp is easy to install, and makes deploying usable common lisp libraries MUCH easier.

By running the following script I will load and test the buildnode system using only libraries pulled from quicklisp.~$ sbcl --script ~/run-builds.lisp --test-system buildnode --quicklisp-only

I have just pushed three Common Lisp libraries to my github account. We have been using Buildnode and TALCL internally for quite some time and CSS-Selectors was something of a learning project I just wrote.

Buildnode

Buildnode is a library to make working with CXML DOM documents and nodes easier. It smoothes some of the DOM interfaces to be a bit nicer and more in line with common lisp conventions. We use buildnode primarily to generate the output of webpages hooked up to our extensively modified UCW lisp web server. We also use it to generate excel spreadsheet XML and google earth KML. It facilitates writing small generation functions that can be built up and combined in any way. We also use it to generate “tag” packages which are a package of functions that build the XML tree for us (see the example).

Primary Features

Iterate drivers for the dom (in-dom-children, in-dom-parents, and in-dom)

DOM manipulation functions such as set-attribute, add-children etc that return the node they are manipulating to ease stringing many calls together and then appending the result to the dom

TAG Packages that make a library of functions for interacting with a specific XML dialect

TALCL

TALCL is a branch of ARNESI YACLML/UCW Template Attribute Language, which in turn was branch of Template Attribute Language originally developed in python for Zope. We think that this version of TAL is much improved over the one originally shipped with UCW by being simpler to use, has better integration with the lisp environment, and simpler evaluation rules. TALCL is divorced entirely from the UCW/ARNESI/YACLML stack and should be a usable choice for any templating need (though certainly specializing in XML templating). We use this library for HTML templates that our designers can work with as well as processing both plain text and HTML email templates in our internal billing software. Examples can be found in the repository.

CSS-Selectors

CSS-Selectors is a library that implements a parser for css-selectors level 3 and provides a compiler that can compile node-matcher functions from these selectors. It also provides a query function (much like jQuery), that can be used to retrieve a list of matching nodes from the dom. I use this for selecting dom nodes to manipulate from the output of an (unreleased) form controls library and for manipulating and pulling information from DOM documents. If static, compilation of CSS-selectors into node-matcher functions occurs at compile time.