This guest post is contributed by Martyn
Loughran, who works at New
Bamboo in London where he builds some very
cool apps like Pusher. Pusher is a web service
which finally makes web push easy using the brand new WebSocket
protocol. When he’s not hacking away at the next great thing, he’s most
likely to be found behind a cello; hacking in this context is less
desirable. Alternatively find him on twitter as
@mloughran where he tweets, reluctantly.

An introduction to eventmachine, and how to avoid callback spaghetti

Evented programming has become cool lately, largely due to the very elegant
node.js project. However, we’ve been evented in the Ruby
world for many years thanks to eventmachine, which adds evented IO to Ruby.
Writing evented code is often viewed as ‘hard’ or ‘back to front’, but it can
actually be very elegant. You just need a few tricks up your sleeve.

One of the big challenges is understanding how to create clean
abstractions. Since the structure of the code is so different to what
you’re probably used to, you can quickly find yourself tied up in
callback spaghetti. In this blog post I will explain some common
patterns you can use while we combine the Twitter streaming API,
Google’s language API, and a WebSocket server. No spaghetti is required,
promise!

After getting started with eventmachine, we’ll discuss two common
abstractions. Firstly deferrable objects which are like method calls
which return asynchrnously. Secondly, how to abstract code that can
trigger multiple events. Finally, we’ll add a WebSocket server into the
same process to show off doing IO concurrently.

Getting started with eventmachine

Let’s start by installing eventmachine and checking that the basics
work:

gem install eventmachine

You can run this ruby test1.rb as you normally would, but you’ll need
to kill it when you’ve had enough.

So how does this work? After requiring eventmachine, we call
EventMachine.run, passing a block. The best advice for now is to
completely ignore this and just focus on what’s inside the block,
eventmachine can’t work without it. (If you’re curious, read up on the
reactor pattern.) Inside
the EventMachine.run block we call add_periodic_timer, passing
another block. This tells eventmachine to fire an event every 1s, at
which point it should call the block. This is the first of many
interesting blocks. It’s known a callback.

You’re probably thinking you could have done this much easier with
loop { puts 'hi'; sleep 1 }, and you’d be right! It’s gets better, I
promise.

Using eventmachine for network IO

Efficient IO is the whole point of eventmachine, but you need to
understand one thing. When you’re doing any kind of network IO with
eventmachine, you must use eventmachine, or a library which uses
eventmachine under the hood (you can usually spot them on github because
they start with em-). You therefore need to be very careful when for
example picking gems which talk to databases, apis, etc.

If you don’t do this you’ll block the reactor, which basically means
that eventmachine will not be able to trigger any more events until the
IO operation completes. So for example, if you used Net::HTTP from the
standard library to request a URL which took 10s to return, the periodic
timer you added earlier wouldn’t fire till it finished. You’ve thrown
concurrency out of the window!

So let’s talk about the HTTP client. Although eventmachine actually
comes with two different HTTP clients, both have issues, and it’s
generally recommended to ignore them and install the very capable
[em-http-request]() instead

gem install em-http-request

Let’s make a quick http request to check that it works (note that EM is
an alias for EventMachine, and by corollary I dislike typing):

Again we’re attaching a callback which is called once the request has
completed. We’re attaching it in a slightly different way to the timer
above, which we’ll discuss next.

Abstracting code that has a success or failure case

When designing APIs it’s extremely common to need to differentiate
between successful and failure responses. In Ruby, two common ways to do
this are to return nil, or to raise an exception
(ActiveRecord::NotFound for example). Eventmachine provides quite an
elegant abstraction for this: the deferrable.

A deferrable is an object to which you may attach a success and a
failure callback, slightly confusingly named callback and errback
respectively. If you’re so inclined you might like to look at the
code,
since there is not much to it.

The call to HttpRequest#get we looked at earlier actually returns a
deferrable (in fact it returns an instance of EM::HttpClient which
mixes in the EM::Deferrable module). It is also quite common to make
use of the EM::DefaultDeferrable which you can use standalone.

As an excuse to use a deferrable ourselves I’ve decided that it would be
a jolly marvelous idea to look up the language of tweets as they arrive
from the twitter streaming API.

Handily, the Google AJAX Language
API allows you
to get the language of any piece of text. It’s designed to be used from
the browser, but we won’t let a small matter like that stop us for now
(although you should if it’s a real application).

When I’m trying out a new API I generally start with HTTParty
(gem install httparty) since it’s just so quick and easy to use.
Warning: you can’t use HTTParty with eventmachine since it uses
Net::HTTP under the covers – this is just for testing!

Before we convert this to use em-http-request (HTTParty uses Net::HTTP
under the covers), let’s wrap it up in a class so we can compare it to
the eventmachine version we’ll write in a minute. I decided to return
nil if the language could not be reliably determined.

This looks pretty different. The important thing is that in all cases
we’ve either called succeed or fail, which we can do since we
included EM::Deferrable. Depending on which one we call, either the
callback or errback block will be executed.

As an exercise you could try to extend this class to retry the api call
up to 3 times on failure. You should be able to maintain exactly the
same external interface, which means that we’ve successfully wrapped up
this complexity in an object and can now forget how it works!

Abstracting code that returns multiple events multiple times

Now we arrive at territory where eventmachine really shines.

We’ll build a client that connects to Twitter’s streaming api, and emits
events every time a tweet arrives.

To use Twitter’s streaming
API you just need to open a
long lived HTTP connection to stream.twitter.com, and wait for the
deluge to arrive. Again we’ll use em-http-request.

Connecting to the API, and listening to all tweets that mention
newtwitter is as easy as doing:

Although this returns a deferrable (which remember triggers its callback
when the request completes), it’s actually got another trick up it’s
sleeve. We can also register to be notified every time new data is
received:

http.stream do |chunk|
# This chunk may contain one or more tweets separated by \r\n
end

It works! Now say we wanted to lookup the language of each tweet using
the class we built earlier. We could do this by adding further to our
stream method, however this is the road to callback hell (and just
imagine what it would be like if we hadn’t abstracted the language
detection!).

Earlier we used a deferrable to abstract code that either succeeded or
failed asynchronously. Another common technique in a lot of eventmachine
code is to provide an onevent callback system. For example, it would
be great if we could have an interface like this: