Queues (Not the British kind)

Apparently, us Brits are well known for queing when it comes to shopping but i’ve not seen much evidence of this in our software. When it comes to queues of the messaging variety we’re probably not that much different from other software developers around the world. Message queues can give our applications some very desirable properties and if they can help make our lives as developers easier then I think they are something we all could and probably should take more interest in. For a lot of developers though queues remain unused and often not even considered. This is defintiely something that needs to change but for that to happen we need to explain why. So in what ways can queues help us?

Most web projects I’ve seen in the real world receive a request from a user and process it real time before returning a response to the user. As the application grows larger, more components get introduced, touching more layers, perhaps calling a web service, and generally causing more work to be carried out during each request. As the load per request increases multiplied by the number of concurrent users the performance of the site degrades. Not only that but the higher the number of components taking part in each request the bigger the chance of a failure which when it happens usually results in a transaction being rolled back, the error logged and some apology given to the user. So, here we have three problems: load, reliability, and performance all of which queues can help us solve.

When a request arrives at your server instead of processing that request immediately you could instead create a message capturing the parameters passed to you and place that message on a queue. You immediately return from the request method body and processing is handed off asynchronously to a component that knows how to handle the message. Your web server does nothing more than capture requests and the responsibility for processing it is handed to an application server instead. This gives you an important benefit, that of time. The requests can come in now as fast as your web server can place messages on the queue which means increased throughput and from the user’s point of view your web site has just become noticably faster.

One by one, the messages are taken from the queue and now perform the work that previously was being done in that tight request/response cycle. Because you’ve already taken the decision not to respond to the user straight away you’ve bought yourself the ability to handle the request in any number of ways. You can distribute the load to multiple app servers. You can simplify some of the interactions that take place by perhaps moving them to yet more queues for things such as sending an email. And if things do go wrong, you can place the message on an error queue, perhaps have an administrator get involved, solve what went wrong and then play the message again. Eventually, the user gets a notification of his request and is none the wiser to what we had to do to process it. In this way we should hardly ever have to present the user with an error page and an apology.

The obvious trade-off is that messages are asynchronous and therefore give us a different problem if we feel we need to show the user something immediately but with a little thought we can often find ways to handle this. Given one particular web application I experienced in a previous job this particular problem would have been a small price to pay in exchange for introducing a queue that would have solved many of its issues but it can be a difficult job to get people thinking beyond the request response mindset. The bottom line is that if you don’t already have queues and messaging as part of your toolbox then you really need to. They can help you solve problems in ways you may not have previously thought of and will provide a degree of robustness, durability, and reliability in your applications that you may have never experienced before.

Of course queues have many more uses especially when it comes to integrating between applications in enterprise scenarios and if you want to really get into the detail then I thoroughly recommend the brilliant book Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe and Bobby Woolf. For what many would consider a quite difficult topic the book is a surprisingly easy read and will teach you so much.