Real time notifications with PHP

Real time communications are cool, isn’t it? Something impossible to do five years ago now (or almost impossible) is already available. Nowadays we have two possible solutions. WebSockets and Comet. WebSockets are probably the best solution but they’ve got two mayor problems:

Not all browsers support them.

Not all proxy servers allows the communications with websokets.

Because of that I prefer to use comet (at least now). It’s not as good as websockets but pretty straightforward ant it works (even on IE). Now I’m going to explain a little script that I’ve got to perform a comet communications, made with PHP. Probably it’s not a good idea to use it in a high traffic site, but it works like a charm in a small intranet. If you want to use comet in a high traffic site maybe you need have a look to Tornado, twisted, node.js or other comet dedicated servers.

Normally when we are speaking about real-time communications, all the people are thinking about a chat application. I want to build a simpler application. A want to detect when someone clicks on a link. Because of that I will need a combination of HTML, PHP and JavaScript. Let’s start:

For the example I’ll use jquery library, so we need to include the library in our HTML file. It will be a blend of JavaScrip and PHP:

As you can see in my example I’ve created a personal protocol for the communications between the client (js at browser), and the server (PHP). It’s a simple one. If you’re looking for a “standard” protocol maybe you need have a look to bayeux protocol from Dojo people.

Let me explain a little bit the usage of the script:

In the HTML page we start the listener (NovComet.subscribe).

We can subscribe to as many events we want (OK it depends on our resources)

When we subscribe to one event we pass a callback function to be triggered.

When we subscribe to the event, we pass the current timestamp to the server.

Client side script (js with jquery) will call to server-side script (PHP) with the timestamp and will wait until server finish.

Server side script will answer when even timestamp changes (someone has published the event)

Server side will no keep waiting forever. If nobody publish the event, server will answer after a pre-selected timeout

client side script will repeat the process again and again.

There’s something really important with this technique. Our server-side event check need to be as simpler as we can. We cannot execute a SQL query for example (our sysadmin will kill us if we do it). We need to bear in mind that this check will be performed again and again per user, because of that it must be as light as we can. In this example we are checking the last modification date of a file (filemtime). Another good solution is to use a memcached database and check a value.

For the test I’ve also created a publishing script (NovComet.publish). This is the simple part. We only call a server-side script that touch the event file (changing the last modification date), triggering the event.

Now I’m going to explain what we can see on the firebug console:

The first iteration nothing happens. 200 OK Http code after the time-out set in the PHP script

As we can see here the script returns a JSON with s=0 (nothing happens)

Now we publish an event. Script returns a 200 OK but now the JSON is different. s=1 and the time-stamp of the event

Our callback has been triggered

And next iteration waiting

And that’s all. Simple and useful. But remember, you must take care if you are using this solution within a high traffic site. What do you think? Do you use lazy comet with PHP in production servers or would you rather another solution?

Nice write up. You’re absolutely correct: we deployed this technique on a high traffic website and found the server frequently bottlenecking. We ended up using writing our own custom little server to handle the requests, as well as some back end caching to prevent unnecessary database calls.

I must admit I didn’t use it in a hight traffic site but it works like a charm in a medium intranet. I avoid any database connection in the comet script and it works without significant server load. BTW I’ve got in my mind a similar script using node.js instead of PHP.

With this example you are still just doing short polling. The difference is that you have moved the polling from the client to the server. If you use a more event driven model, you can greatly improve performance. You won’t have blackout periods between polling memcached or while sending the data out to the client. I’ve written a small PHP HTTP/WebSocket server that allows you to take incoming messages from your main server and distribute them to many client connections at once. The smaller server (which is based off of Tornado and called WaterSpout) listens for events from the main server and then distributes them immediately to the proper client connections. It uses WebSockets to ensure a constant connection but will fail over to long polling if WebSockets aren’t available.

If you are going to be at Tek11 this year, stop by my talk on Realtime Communications. I’d love to chat about the challenges you’ve faced and the way you went about solving them.

I know. I need to use websockets to improve the performance. The main problem I’ve got with websockets is the browser (only a few of them) and proxy servers (they normally kill websockes). But this solution works without any problem within a small/medium intranet (the number of users is limited). My sysadmin did not kill me (yet), and the server load isn’t altered. I’ve taken care with databases (no db connection at serverside), and I check the modification date of files located in RAM (/dev/shm). The apache connections are limited by a timeout (they’re not alive forever). OK, you’re right. That’s still short polling and it’s not a “real” real-time, but it looks like real-time. Good part: works with all browsers, and I don’t need to take care about internet proxies. Bad part: if I scale it, my sysadmin will definitely kill me. Now I’m playing with node.js + socket.io library (in order to use websockets)

I’ve seen your talk about it at phpconference2010 (BCN). Definitely your talk inspired me a lot. In fact I’m the one who spoke with you (at the end of your great talk) at about the problems with proxy servers and websokets ;).

Really interesting. Do you think Facebook/Gmail using the same way like this? I know they are using long-polling/comet. Is this the same? Why these two website still works like a charm? They have billions of users… Because of their amazing servers?

No. This technique is not suitable for big traffic. Your server load will raise to hell, and that’s too expensive. I’ve wrote a second part of the post (https://gonzalo123.wordpress.com/2011/05/23/real-time-notifications-part-ii-now-with-node-js-and-socket-io/) doing the same with node.js and websockets/socket.io. That’s a better solution if you need to scale. FB/Gmail/Twitter’s servers aren’t amazing. The only amazing thing is the number of servers :). The solution of this post “works” but the solution of the second post works too and scales too. The “problem” with the second solution is that you need one extra server for Real Time communications.

The problem with the PHP’s solution (The first post) is the IDLE mechanism used (the sleep function). Node uses functional programming and its way to send process to IDLE is brilliant (with callbacks). So different ways to solve the same problem, wiht its pro and cons.

I don’t think so. They published couple of years ago one similar open source project called Tornado (python based). It’s similar than Twisted (python too). As far as I know FB uses python in the back-end and probably they have one customized version of Tornado. The idea is the same than node.js.

You are right. They are using tornado. How do I implement it on tornado? or I can simply use node.js? since i tried facebook on firebugs and found that they were using long-polling php as well. Should I set up another server to get node.js/tornado work?

hi i am using your script but not getting the correct manner to use it.While i used it i have got some error like / comet.php

Warning: touch() [function.touch]: Unable to create file /dev/customAlert2.comet because No such file or directory in C:\xampp\htdocs\realtime\NovComet.php on line 73
false

in firebug response box.
Actually i want to make real time notification in my php website but not geeting any idea.as go through the google i got your script but not working it too for me please help me.
thanks

In the example I’m using /dev/shm to store files.That’s because it’s virtual file system mounted in memory (Linux systems). As I see within your comment you are using Windows, so you need to change the path of the files (hardcoded in DEFAULT_COMET_PATH constant). Anyway I recommend to use another alternatives with node.js or react (with PHP). The approach of this post works, but it will throw problems if you need to scale. (you can see examples in newer posts here for example)

I sad that it was “almost” impossible. In fact this implementation is also PHP and plain js and it was available 5 years ago, but nowadays websockets plays in another league. Without websockets we could do it using exotic implementations (like the example of this post) or “emulate” real time with timers. The problems. With a limited number of users those implementations was viable. I have something similar than this post in production for for years in a intranet and it works like a charm indeed, but I now that that if I need to scale it I will have serious problems. Nothing compared to the simplicity of a node.js server with socket.io or even PHP with rachet.

hi, your example is working great “out of the box”,
I was wondering if it can support a publish action publish( ” my message”,”subscribed channel”) and subscribe (“subscribed channel” alert (“message” )?
thanks
jean-louis

Of course not :). I use to use this tricky technique before websockets become mainstream. Nowadays I use socket.io for real time notifications. Anyway If I’d need to build a websocket system from scratch I don’t know if I’ll use socket.io or crossbar.io