Monday, April 16, 2007

RLisp was a wonderful idea, unfortunately its first implementation was a very slow pure-interpreter, and it was too difficult to retrofit good performance into it.
The first version of RLisp even evaluated macros each time they were encountered, making them in practice "FEXPR"s instead of macros.
I was able to improve RLisp performance a bit. The biggest gain was making sure macros expand just once. Other optimizations resulted in only minor improvement, and the code was quickly turning into a unreadable mess.
The solution was a major rewrite. In the new implementation RLisp code is compiled to something that looks a lot like Lua 5.0's virtual machine code, and then to pure Ruby. Some of Lua virtual machine's ideas like flat closures are simply awesome and greatly simplify the implementation.
The code is much cleaner, and much faster. In "x times slower than Ruby" benchmarks:

Benchmark

ary

hello

ackermann

Early interpreter

x96.1

x37.8

x1027.0

Optimized interpreter

x35.5

x58.2

x53.1

Compiler

x5.3

x6.7

x7.5

Compiled code

RLisp compiler changed a lot since the teaser.
Now to create functions it uses Object#function_* and Proc.new instead of Function objects. Function objects were more elegant, but they could only be used for free-standing functions, not for methods - inside such function self was a closure.
The new form makes it possible to use RLisp (fn ...) as methods, instance_eval them and so on. Unfortunately as Proc.new{|*args,&blk| ... } is a syntax error in Ruby 1.8, it's impossible to create RLisp functions which take Ruby-compatible block argument. Fortunately you can still call such functions from RLisp code, and RLisp functions can take Proc objects as normal arguments.
A few examples.

If RLisp can determine that a variable is constant or isn't used within nested function, it doesn't create Variable objects for it, and performs various optimizations based on such information. Of course Variable objects are created when needed, like in the following code which creates a counter function. A counter function called with an argument increases the internal counter and returns the old value.

Access to Ruby

RLisp can access Ruby with (ruby-eval "Ruby code") function. All unknown uppercase variables are also assumed to be Ruby constants (nested constants like WEBrick::HTTPServer not supported yet).
This makes using most Ruby libraries from RLisp pretty straightforward.

RLisp can be run in two modes. To execute a standalone program use rlisp.rb program.rl. To run interactive REPL use rlisp.rb -i. It uses readline for input and prints out each expression as it is evaluated. Like every decent REPL, it captures exceptions without exiting.

Other options are -n (don't use RLisp standard library), -r (force recompilation of standard library), and -h (display help message).

The future

You can download RLisp from RLisp website. It's licenced under BSD/MIT-like licence, has reasonable performance (CPU-wise 5x slower than Ruby - perfectly reasonable for database-bound or network-bound programs), and a lot of libraries (that is - it can use most Ruby libraries out of the box).
The code is rather clean, and reasonably tested (test code to code ratio 0.96:1). No major program was written in RLisp so far, so a few misdesigns and major bugs are certainly present.
What I'd like to do is create a new backend for RLisp. For now Lua-like VM opcodes are compiled to Ruby, but they're so simple that they could be compiled to C, x86 assembly, or LLVM and still stay fully compatible with Ruby runtime. That would probably make RLisp somewhat faster than Ruby, as Ruby's slowest part is eval. Ruby backend could probably be made faster too without too much work.
It would be awesome to do add at least limited tail-recursion optimization to RLisp.
Have fun.

My software

Creative Commons

Unless otherwise expressly stated, all original material of whatever nature created by Tomasz Węgrzanowski and included in this blog, is licensed under a Creative Commons License. It is also licensed under GFDL (for Wikipedia compatibility).