Trade Secrets Of A Developer / Entrepreneur

Why Messaging? Why Not Have The Web Server Handle The Request?

A reader of my RabbitMQ Patterns email course recently asked a question about using messaging systems within an web application. Specifically, they wanted to know why you would use a request/reply pattern over a messaging server, instead of just handling the request within the HTTP call and the web server, itself.

The short answer is, “it depends”.

There are times when the request can and should be handled by the web server, directly. Loading a blog post to view? Yeah, that’s probably a good call for the web server to handle. But there are also times when it doesn’t make sense for the web server to handle the workload, directly.

The following is a short conversation that took place via email, regarding messaging vs HTTP.

Shifting The Workload

Them:

What are the benefits of using a request / response queue over just doing the work as part of http request?

For fire and forget messages I can see the performance benefits…

Is it just being able to shift the work load to another service? Or abstract some complexity of the work?

Me:

That’s the core of it, really – shifting the workload to another service.

If you have an external system that contains the information you need, or can do the work within a few hundred milliseconds, it might make sense to do request/reply.

Scheduled Code Example

Me, continued:

For example, I have a system that uses a node.js module for running tasks at specific times. It’s a bit like a cron job, but it runs entirely in node.

This module lives in a back-end service that runs in it’s own process. I used a separate process because this service has nothing to do with handling HTTP requests and should not live in the HTTP context or web server, directly.

But, my web app needs to get a list of what’s running next, and display them on a particular page. So how do I do that? What are the options?

Read The Database Directly

I could read the database collection that the scheduler module uses – it’s in the same database that the web app uses. But, that would be a backdoor into the functionality the module provides. I would have to re-create the logic of the scheduler module within my web app, to translate the data into something meaningful for the user.

No, thanks.

A New Schedule Module Instance

I could create new instance of the scheduler module in my web app. But this would create two sources of the truth on what is running next, and they would be in conflict with each other.

Each instance would assume it owns the data in the database (because the instances don’t know about each other). This would cause problems when one updates the database and the other tries to update it again.

Again: No, thanks.

Request/Reply

The solution, in this case, was to use request/reply to get the list of items from the existing back-end service. That way I have a single source of the truth for the scheduled items, and don’t have to re-create logic from an existing module.

Decoupling The Database

There’s a saying I heard a long time ago, “a database is not an integration layer.”

I don’t remember where I first heard that, but it continues to ring true after many years.

I could have tried to use the database as an integration layer, but that would have added complexity and potential problems to the system.

It made more sense for me to use messaging in this scenario, and allow the one source of truth on the upcoming schedules to be that one source of truth.

Beyond Reading: Writes With Messaging

There are far more examples of what can (and should) be done with messaging, when handling HTTP requests. The list is nearly endless, and can go so far as to say database writes should be handled through messages!

In my RabbitMQ For Developers training package, I spoke with Anders Ljusberg about Event Sourcing and messaging systems for database writes.

It was an enlightening conversation, to hear about the need for pushing a database write to a back-end service, and how this affects the over-all architecture of a web application.

About Derick

Derick Bailey is a developer, entrepreneur, author, speaker and technology leader in central Texas (north of Austin). He's been a professional developer since the late 90’s, and has been writing code since the late 80’s. Derick has built software for organizations of all shapes and sizes, including contributions to Microsoft's MDSN library, running several very highly regarded open source projects, creating software solutions for large financial organizations, healthcare orgnaizations, world-class airlines, the U.S. government, and more. These days, Derick spends most of his time working on content for his own entrepreneurial efforts at WatchMeCode.net, playing video games when he gets a chance, and writing code for for his few remaining clients. You can reach Derick at DerickBailey.com or on twitter, @derickbailey.