Deliver real-time information to your users using node.js

July 26th 2010

You must have already heard about node.js, if not, it’s an evented input/output server based on Google V8 engine. That means that as soon as the server receives information it can respond and send relevant information back to your front-end. You can then treat this information the way you want. What is so cool about it is that if you know javascript you can code a node.js server and it really is a lot of fun!

In this article, I’ll cover the basics to be able to build a simple server that will respond to an “admin” user input and then broadcast a message to all users connected on the website. Based on that, you’re then free to apply it to whatever needs you may have.

Please note that some basic server knowledge is required, like installing software with command line and forward requests to specific ports.

The demo

First open the live feed demo page, then open the live feed admin page. Try to resize them so you can see both, then click on the display buttons on the admin page, watch the magic happen on the live feed page!

The basics

How does that real-time thing work? Well it’s actually quite simple. Every time a user connects to my demo page, I open an ajax connection, that ajax connection wait for the server to send data. Once it receives data, it’s processed, the connection is closed and a new one is opened. This can be called Comet, long polling, ajax push, HTTP Streaming and a bunch of other names. You can also do long-polling in PHP but since node.js is a dedicated server it does a better job at handling the load.

The setup

If you’ve never installed node.js on your server, you first need to do it. The documentation is here. Make sure you have g++ installed on your server. Simply run: “sudo apt-get install g++” if you’re on Ubuntu, it’ll either tell you it’s up to date or install it.

The ITEMS_BACKLOG is the max number of items the server keeps in memory, this way you can make sur it never runs out of memory, you’ll see where it’s being used later.

I then declare my urlMap, these are the URLs the server listens to and the code to execute upon calling. It’s fairly straight forward, there’s one very important thing you need to know tho, it’s that since node.js runs on a different port than your normal webserver (a webserver usually runs on port 80), you need to forward the calls on those URLs to your node.js server port.

In my case, I use NGINX as my webserver and node.js runs on port 8001. Here’s what my NGINX configuration file looks like:

You see that I redirect both URLs in my “urlMap” to the node.js server port. Might not be the most efficient way but since you cannot do cross-domain posting in ajax, it’s the only solution I found for now.

Then comes the meat, the server itself, it’s fairly easy to create, you just need to invoke the “createServer” method and make it listen to a specific port.

http.createServer(function (req, res) {
...
}).listen(8001);

All requests and responses will be processed in here, so you need some code in there right? Here it is:

So the first thing I do is go get the function that need to be invoked in my urlMap and assign it to my handler. Then depending on the type of request, I either get the posted data and send a response or just straight invoke the handler of the URL called and send a response.

Now you’ll see that in my urlMap I call the feed object, this object has 2 methods, one for querying and one for appending content to the feed.

What you need to do on the client side is very simple, you only need to start an ajax call that restarts when it receive a response:

function longPoll_feed () {
//make another request
$.ajax({
cache: false,
dataType: 'json',
type: "GET",
url: "/real_time_feed",
error: function () {
//don't flood the servers on error, wait 10 seconds before retrying
setTimeout(longPoll_feed, 10*1000);
},
success: function (json) {
display_event(json);
//if everything went well, begin another request immediately
//the server will take a long time to respond
//how long? well, it will wait until there is another message
//and then it will return it to us and close the connection.
//since the connection is closed when we get data, we longPoll again
longPoll_feed();
}
});
}

So that ajax call listens to what is sent on the “real_time_feed” url. Once it receives data, it processes them.

Since it’s JSON that is returned to my views, I can then treat it the way I want. This file need to be included in the HTML file that will receive the live content.

The big merge

Ok so now you have your server javascript file and your client javascript file. The first thing you need to do is create an HTML page in which you’ll include the client.js file, I my case, it’s the index.html file.

Now for the server.js file, you actually need to connect via command line and start the node.js server like so:

And finally, to be able to send content to the server in real time, you need to POST content to the url “/send_feed_item”. If you remember in my URL map, I have setup this url to append content to my feed. So a simple ajax post should do the trick to post content, that’s what I do in my admin page.

The sources

The possibilities

Let’s just say you watch a live conference on ustream.tv and the speaker mention some website, with node.js he could be able to send you to this website in real time as long as you are connected to his server or you could be watching live news on a website and they mention traffic, you could then be presented a Google traffic map live as they talk about it on the stream.

The possibilites are endless, this post covers only the basics, I’ll let your mind do the rest 😉

Related Posts

If what you were reading interested you, those might too! Check them out!

@Dominic: I am working on a real case project using node, I just can’t disclose the details yet.

As for NGINX, I use it because it’s using a lot less memory than apache. I’ve set my server up so the static files are served by NGINX and the dynamic served by apache. This way I get the low memory footprint and speed of NGINX but don’t loose any of the features apache can provide.

” I’ve set my server up so the static files are served by NGINX and the dynamic served by apache. This way I get the low memory footprint and speed of NGINX but don’t loose any of the features apache can provide.”

How did you set this up? If you were using WordPress or Drupal, is this still achievable?

Great tutorial! I have a question: this work only if client and server are on the same machine, right? I’m wondering if it’s possible to do the same with remote client, using JSONP maybe. Is it possible?

Ok, but your client use an ajax request to retrieve the data so if I use it on a site that is outsite my domain is a same-origin policy violation. Am I wrong? What I’m trying to do is a widget that a user can embed in his webpage and that is updated in real time when I push new data from the node.js server. Dont know if this make sense…

Do you like travelling? So do I. What’s the most annoying thing in travelling? Expenses and time you must spend getting from one point to another. It’s difficult to find solution with time but you can do something with prices. You can choose bilety autokarowe that are really cheap and anyone can afford to buy them.

1. I copied the entire source folder and i navigated to js folder and typed “node server.js”. Now, i browsed to “localhost:8001” and the result i got is “Not Found”. This is because the code has only two checks one is “/real_time_feed” and another is “/send_feed_item”. Do we need to implement for “/index.htm” or “/”, “/admin.html”, “/css/master.css” and “/js/client.js” also? For now i changed server.js to handle those requests as well.

2. server.js is running on localhost and port no 8001. now, how to redirect requests to node? I mean in this case i’m browsing “http://nodejs.no-margin-for-errors.com/”, but how come the node is listening to this?

I don?t even know the way I ended up here, however I assumed this publish was once good. I do not recognize who you’re however definitely you are going to a well-known blogger for those who aren’t already 😉 Cheers!

Great tutorial!
But I got an issues when I tried the source codes on my local machine. I run server.js and view the result on the browser but Not Found appeared on my screen. I don’t know what’s the problem.I’m just a beginner in nodejs and right now we’re planning to develop an app using nodejs.Can you help me with this?Thank you so much!

On the demo, my JS console says:
“Failed to load resource: the server responded with a status of 502 (Bad Gateway)”
“2event.layerX and event.layerY are broken and deprecated in WebKit. They will be removed from the engine in the near future.”
and it doesn’t work. Any idea how come?

hello. Read your source. It look like while(1) read_from_server(); I learn JS two days, but can tell you that this is not JS way. I`m talking about that:
“What you need to do on the client side is very simple, you only need to start an ajax call that restarts when it receive a response”

Best choise (imho) use async_calls from server side.(node.js+now.js+socket.io). What you think about it?
Best regards.Mike.

INK is a new way to own your content. It’s a well known fact, the content you produce can be shared all around the web and in a very large number of ways. One of the most common ways is to simply copy the content and paste it wherever we want it to be shared. […]

prettyLoader is a small (less than 4kb uncompressed) jQuery plugin that aim at making your life easier regarding ajax loader display. You know, when you do an ajax call and you never quite know where to position your loader. prettyLoader takes care of that for you, by default, prettyLoader will “hook” to the jQuery ajax function […]

prettySociable is a jQuery plugin that tries to make sharing fun while being easy to use. It was inspired by the sharing on mashable.com. While ShareThis and AddThis provides a very useful piece code that allow users to share basically anything everywhere, their solution is not the prettiest nor the easiest to use. prettySociable simply […]