Monday, 5 January 2009

There are some Ruby features I like a lot. A handful of them are borrowed from other languages (like lisp or perl), but the Matz found a way of making the language a very nice and powerful combination of them. I've been using Ruby lately and got thinking on how could I implement one of Ruby's most beloved features, blocks, in Python (remember, "imitation is the sincerest form of flattery"). As it turns out, it is not all that hard. Here's what I came up with:

But before that...On Ruby blocks

Blocks in ruby provide a way of creating functions that act on a code block defined later on.An example of blocks in ruby is the following:

array = [1, 2, 3, 4]array.each { |n| puts n ** 2 }14916

According to their site: "A block is like an anonymous function or lambda [and] the variable between pipe characters is the parameter for this block". What's missing from this description is that ruby also provides the syntactic sugar to create functions that receive blocks using the "yield" statement. In other words, it is a way of creating closures and attaching them to other methods. Using closures in ruby comes very easily even for people that donsn't know what a closure is. If we were to implement the previous behaviour ourselves we would do something like this:

class Array def each2 for i in self yield(i) end endend

array = [1, 2, 3, 4]array.each2 { |n| puts n ** 2 }14916

I won't go into details because there is a lot of documentation on ruby blocks out there.

So, onto Python...

DesignThe Python "builtin" that resembles the most to blocks is that of PEP343, the with statement; but I wanted something that immitetad the ruby syntax as much as possible. The with statement is nice, but it doesn't cover all the cases.

So I decided to use a decorator to convert the function that uses the "block" into something that receives the block, inserts it into the namespace, and execute the original function with the block as a corutine.

The idea was something like this:

@receive_blockdef simple_iterate(): for i in [1,2,3]: print block()

@simple_iteratedef _(): return "a"

This copies the Ruby syntax except for the receive_block decorator, but I considered it a reasonable sacrifice.Using "def _()" leaves the function as anonymous and allows you to specify parametrs for the block.Implementing blocks in PythonSo, to implement the syntax I just need to write the receive_block decorator.The objective of this decorator is to convert the block receiving function, A, into another decorator that receives a function, B, introduces B into A's scope and subsequently calls A.The key step is to add the block function to the scope. To do this we use Python's builtin types module. It includes the FunctionType method which creates a function object.

types.FunctionType(func.__code__, scope)

There is more to this method than what we use here, but I won't go into details about the method since we only need the simplest use of it.Once we know this, the decorator is pretty simple:

Another interesting way to do this would have been to add the block variable as a free variable of the function and have the code object reference it. In Python, when a closure is created the free variables are stored in an attribute of the function's code object and it's values are stored in the function itself using the cell type. Take this closure as example:

If anyone, who knows better than I do, cares to provide an implementation using closures I'd love to hear your solutions.

So this is how we implement Ruby-style blocks in Python using decorators. Hope you enjoyed it.

DisclaimerThis is by no means meant to be used in a production environment. Not even in a semi-serious environment. It is just a hack to demonstrate the how this could be done and it hasn't been tested.

Other notes* The code for this project is hosted in http://github.com/nicolaslara/blocks/tree/master* I wanted to implement some of the builtin Ruby functions that use blocks but I didn't have the time. If somebody is up to the task I'd love to see what interesting things could be done with this.* Also, if somebody is willing to improve the code you are more than welcome.* Some of the problems of this code:-It clutters the global namespace.-The word "block" is introduced as a global reserved word.* For some reason blogger doesn't let me edit this site's template in 2009, so I couldn't add code syntax highlighting.

Thanks Alex! I was using <pre> because that's what my syntax highlighter uses but I couldn't add the highlighter 'cuz blogger wouldn't let me edit the template. It's fixed now thanks to a a cool widget by FaziBear

No particular reason. I just wanted to avoid passing the argument explicitly. Note also that this breaks when using other paramenters in the block handling function. We would need some use of partial to evaluate the params.