Wednesday, May 04, 2011

After pondering the results of our message queue shootout, we decided to run with Rabbit MQ. Rabbit ticks all of the boxes, it’s supported (by Spring Source and then VMware ultimately), scales and has the features and performance we need. The RabbitMQ.Client provided by Spring Source is a thin wrapper that quite faithfully exposes the AMQP protocol, so it expects messages as byte arrays.

For the shootout tests spraying byte arrays around was fine, but in the real world, we want our messages to be .NET types. I also wanted to provide developers with a very simple API that abstracted away the Exchange/Binding/Queue model of AMQP and instead provides a simple publish/subscribe and request/response model. My inspiration was the excellent work done by Dru Sellers and Chris Patterson with MassTransit (the new V2.0 beta is just out).

/// <summary>/// Subscribes to a stream of messages that match a .NET type./// </summary>/// <typeparam name="T">The type to subscribe to</typeparam>/// <param name="subscriptionId">/// A unique identifier for the subscription. Two subscriptions with the same subscriptionId/// and type will get messages delivered in turn. This is useful if you want multiple subscribers/// to load balance a subscription in a round-robin fashion./// </param>/// <param name="onMessage">/// The action to run when a message arrives./// </param>void Subscribe<T>(string subscriptionId, Action<T> onMessage);

This creates a queue named ‘test_<message type>’ and binds it to the message type’s exchange. When a message is received it is passed to the Action<T> delegate. If there are more than one subscribers to the same message type named ‘test’, Rabbit will hand out the messages in a round-robin fashion, so you get simple load balancing out of the box. Subscribers to the same message type, but with different names will each get a copy of the message, as you’d expect.

The second messaging pattern is an asynchronous RPC. You can call a remote service like this:

This first creates a new temporary queue for the TestResponseMessage. It then publishes the TestRequestMessage with a return address to the temporary queue. When the TestResponseMessage is received, it passes it to the Action<T> delegate. RabbitMQ happily creates temporary queues and provides a return address header, so this was very easy to implement.

This creates a subscription for the TestRequestMessage. When a message is received, the Func<TRequest, TResponse> delegate is passed the request and returns the response. The response message is then published to the temporary client queue.

Once again, scaling RPC servers is simply a question of running up new instances. Rabbit will automatically distribute messages to them.

The features of AMQP (and Rabbit) make creating this kind of API a breeze. Check it out and let me know what you think.

EasyNetQ was written to provide the messaging infrastructure for 15below.com. We provide integration systems for many of the world's leading airlines, including Virgin, Ryanair, JetBlue and about 30 others. We typically see transaction volumes in the millions across complex business processes. RabbitMQ and EasyNetQ have worked pretty flawlessly in the year or so we've been using them. I'd say that's 90% down to RabbitMQ which has simply been an ultra-reliable black box, but there haven't been any major bugs with EasyNetQ either.