February 18th, 2007

I’ve recently found myself describing single objects yielded by
iterators as “magic pens.” Though it may sound more psychodelic than
technical, it’s actually proven to be a good way to explain these
objects and give people a feel for why an iterator would only yield
one object in the first place.

I don’t mean cases where you iterate through a collection that happens
to have only one element. I mean methods that, by design, yield one
and only one object, every time they’re run.

The very word “iteration” evokes collections, and it’s not surprising
that I’ve seen puzzlement over iterator constructs like this one:

The create_table method is being supplied with a block, so it’s an
iterator. (Ask any method: iterator? and it will tell you true if
you’ve given it a block, even if it never yields to it.) But what is
it iterating over, or through? How many times? Once? Why bother
iterating, then? Why not just call a method and use the return value?

The answer—the admittedly very high-level answer, but the one I’ve
found most effective in nailing into place exactly what’s going on—is that create_table works by handing you a magic pen. The pen has
magic only for the duration of the code block. While you have the
magic pen, you use it to do things you wouldn’t normally do, like
bring about changes in your database tables and columns. (Not the
records (though those too, if you wish): the tables and columns themselves.)

The magic pen idiom is very elegant, for at least two reasons.

First, it throws the spotlight on what you’re doing. The delimitation
of the code block, during which you have the pen, puts the specialness
of the process in high relief. You have to get the pen to do what it
needs to, all in one place and time. (You might say that the code block is itself a magic “pen”—that is, a magic enclosure….)

Second, the magic pen idiom saves you trouble. Think about the method
that hands you the pen. At some point, it has to yield to your code
block. But before and after it yields, it can do any amount of set up
and tear down—which, consequently, you don’t have to worry about.

Let’s look at an illustration of this second point up close.

Inside a magic pen

The magic pen you get from create_table is actually an instance of the
class TableDefinition. You can see exactly how create_table hands you
the magic pen if you look at the source code for the first segment of
the method create_table, which is in schema_statements.rb (comments
added):

The yield triggers execution of your code block, and the local
variable t is assigned to the TableDefinition object that’s been
yielded. You need this object because it’s the one that has a
“column” method. And it’s your magic pen.

The magic itself (the process of changing the database) happens
back in create_table, after your code block ends. Here’s the rest of
create_table:

The magic pen is “map”, which is a Mapper object. It’s by handling
this object that you bring about changes in the routing tables. Like
the migration magic pen, the routing magic pen is yours only for a
time. But during that time, it gives you a lot of power. When you
write named routes and resource routes, you see even more of the
power:

# Create a method called profile_url that generates a URL
# of the form profile/id which is bound to the user/show
# action:
map.profile 'profile/:id',
:controller => "user",
:action => "show"
# Generate a full complement of RESTful routes and associated
# URL generation methods for the accounts resource:
map.resources :accounts

This magic pen has a lot of ink in it.

Last but not least, let’s look at the extremely useful filehandle
magic pen in Ruby.

Ruby’s filehandle magic pen

You can, if you wish, deal with file-writing procedurally, like this:

fh = File.open("myfile", "w")
fh.puts "Content of file!"
fh.close

And in this scenario, you do indeed get a magic pen. But it’s not
very dignified for the magician to have to sweep the stage after the
show—as in fh.close. Doing it the magic pen way has the effect of
making Ruby do the housekeeping:

File.open("myfile", "w") do |fh|
fh.puts "Content of file!"
end

The open method yields a filehandle. Then, after the code block
finishes, control returns to the method, where the filehandle gets
closed for you. You don’t have to worry about it.

The magic pen idiom distills the magic into the pen, by offloading
the housekeeping onto the method that yields the pen in the first
place (create_table, draw, open). Conversely, the fact that Ruby
makes this kind of offloading so easy is what makes magic pen
scenarios so easy.

So when you see an iterator, don’t assume it’s handling a collection.
It may just be setting up a magic pen experience for you.