Introduction

While doing a code review for a friend of mine, I came across an interesting technique for getting a web service to “push” an event or message back to the client when something occurs. We all know that the HTTP is a request/response protocol. The client makes a request, a socket is opened, the server allocates a thread for the request, the request is processed, the results are returned, and the socket is closed. In the code that I saw, and reworked/improved for this example, web services are used to communicate events or messages between the request and the response.

Think of a chat room application entirely handled by web services. The various “events” or messages that would have to be passed between users include “user has logged in”, “user has logged out”, and the actual text messages sent.

Background

Normally, this would be handled by writing some TCP socket based application that maintains sockets between the server and the clients. The advantage to using a web service is that you don’t have to manage the sockets or the thread pool yourself. You also don’t have to worry about configuring every firewall on every client machine to open a port for you; HTTP runs on port 80, and is not blocked. This is known as HTTP Tunneling. Your clients can communicate with each other through HTTP, without firewalls and anti-viral applications treating them like some Trojan horse.

The major disadvantage of this technique, and we’ll see this further in the article, is that it requires holding on to a thread pool thread on the server. Although the tuning and performance aspects of this problem are beyond the scope of this article, suffice it to say that this technique may not scale very well. Still, if the application is served by a dedicated server with a predetermined amount of clients, and the thread pool configuration on the server is properly done, this solution can be compelling.

Besides practical usage, this article showcases some of the challenges and techniques in writing multi-threaded applications. Some of the .NET threading classes that I have used in the example can be replaced with others. I believe the ones chosen best fit the example for both functional and performance reasons, but feel free to experiment with various constructs. At the end of this article, I’ve included links to various sites that deal with multi-threading.

Using the Code

OK, so how do I get a web service to notify the client when something occurs on another client or on the server itself? As I mentioned earlier, web services use a request/response protocol. How do we get it to push back a message? The answer is, we don’t. We simulate a push back by stopping the request on the server before it returns to the client. This process simulates “listening”. When an event occurs, such as another client sending a message through another web method, our listening web method is released, returning an “event” object that represents the type of event/message that occurred and containing any data that pertains to it.

Initial state – Everyone’s listening for an event from someone

An event occurs – A listen request is released and a response occurs

This is possible since a web service can be called asynchronously from a form without locking up the form’s main thread. A callback event handler is used to process the incoming event from the server.

Calling a web service asynchronously is made easy in .NET 2.0 since every web method you create generates two methods and an event on the client proxy.

For instance, if you were to create a web method called HelloWorld, you’ll see that the client proxy for the web service has a HelloWorld method, a HelloWorldAsynch method, and a HelloWorldCompleted event. The first method, as you probably know, relays the execution request to the server and blocks the calling client thread until a response occurs. It is said to be synchronous. HelloWorldAsynch acts as a fire-and-forget method. You call it from your client code, but it never blocks the thread. Your form is free to continue processing without having to wait for the web method to return a value or a response. When the response arrives, it fires off the HelloWorldCompleted event which can be handled by your client code.

The Listen method described above works in this manner. When the form loads, it calls for a Listen using the asynchronous version of the method. When an event occurs on the server that needs to be reported to the client, the ListenCompleted event fires off. This event is handled with a method that processes the “event”. It then calls Listen again to be ready for the next event.

The ListenCompleted event comes complete with a ListenCompletedEventArgs object, also generated by the IDE. It contains an object of the same type as returned by the Listen web method.

Since the Listen web method returns an “EventObject” array defined by me, the event argument’s Result property will contain an object of that type. EventObject is the ancestor of three event classes that my client application is interested in listening to and processing. They are as follows: LoginEvent, LoggedOutEvent, and MessageEvent.

Since my sample application is a simple chat room, I want to display all users that are currently logged in, and being told in real time when a new user has logged in, when a logged in user has logged out, and when a message has been sent by someone in the chat room.

I made the Listen web method return an array of events because various events can occur while we were not listening. I.e., while we were processing an event on the client, and before calling the ListenAsync web method, various other events can occur on the server that we may have missed (I’ll get into the caching mechanism of these events further down the line). When we listen again, we want to receive all the missed events in one shot. Hence, an array of EventObjects.

The list will be traversed upon the arrival of an event, and each WaitHandle will be released with a Set method on the WaitHandle. So, if we peek at a snippet of code from the SendObjectEvent method that occurs with each event, we’ll see this:

SendObjectEvent then goes into its own wait state that’s released only when all the Listen threads have been released so that it can do some post-event-release housekeeping. This is achieved by the Listen method by decrementing a listener count right before returning the event object to the client. When the count is zero, it signals the SendObjectEvent method through a global waithandle called AllListenersReleased.

That’s the concept in a nutshell. Of course, no multi-threaded solution is ever that simple. But, before I dig deeper into the issues and the solution that cropped up while writing this sample application, it's important to understand the core objective’s mechanism:

Have the Listen method block the thread on its own so that it doesn’t return a response until some event occurs and data pertaining to that event is ready to be returned to the client.

Retain the blocking WaitHandle object for each client application running the Listen web method on a web server thread. The list of WaitHandles are retained in a global list.

SendObjectEvent releases each listening thread when an event occurs that needs to be pushed back to all the listening clients, by issuing a Set on each WaitHandle that we’ve accumulated in the global list. It then blocks/waits until all the Listeners have been released.

The Listen method signals the waiting SendObjectEvent that all listing threads have been released so that it can do some housekeeping such as un-registering the Listener WaitHandles (the client will re-register a WaitHandle on subsequent calls to Listen).

Digging Deeper

Ancillary Issues

The following issues and challenges must be addressed before the core objective is complete:

Synchronization - The Listen web method and the SendEventObject method, which all events call to release the Listeners and pass event specific data, must be synchronized and thread safe.

Can’t start to Listen while sending an event.

Can’t send an event while the Listen method registers its waithandle.

Can’t process more than one event at a time.

Don’t want to register two listener waithandles at the same time.

Event Caching - The web service must cache events until it’s certain that all clients received them. This becomes an issue when events are pushed back while one or more clients aren’t listening due to being busy processing a prior event.

If outstanding events exists for a client, they should be returned immediately upon the subsequent Listen call.

The web service must have a reasonable way to determine if all events have been delivered to all clients. Once that has been determined, the events cache can be cleared. The list of waithandlers can be cleared at this time as well.

Request Timeouts - Request timeouts must be addressed. Before a timeout occurs, we must release a dummy event back to the client. The client in turn recognizes that it received a “non-event” event and simply re-executes a Listen.

Synchronization

There are various locks in the code to ensure that multiple threads do not modify global variables simultaneously. I’ll focus on the locks that synchronize access between the Listen web method and the SendEventObject method that all the “application events” use to push back an event.

Conceptually speaking, listening for, and sending, an event needs to be synchronized. I don’t want to send an event while a Listen method is trying to register its waithandle to the global list. This could cause a race condition were the Listen method hasn’t registered itself for receiving events, but an event has already been released to all the listening threads. Since all registered listening threads are assumed to have received the event, the event cache can be cleared and a client can lose an event.

Furthermore, I don’t want two events processed simultaneously either. Although I can push back more than one event to a client, I want to manage "one event/one release" to all listening clients at a time.

Below are the first few lines of the Listen method. A generic lock object (Global.LockObj) is used in a code block to protect various global variables that are being changed - that has to be done atomically. Within that code block, I enter a loop that checks if an event is being delivered, and blocks the current thread if it is.

I use a global state variable called EventBeingDelivered to determine if a client is in the process of registering a Listen, or if an event is in the process of being sent to all the clients. Since I’m using a “blocking condition flag”, as mentioned in Joseph Albahar’s excellent e-book on Threading in C#, I’ve chosen the Monitor class as the synchronization mechanism between the Listen and the SendEventObject methods.

This is essential since a Monitor.Pulse may have executed indicating that an application event has been processed successfully, but another thread running a SendEventObject may have received execution control prior to the Listen code executing. We, therefore, check the blocking condition flag for its state prior to continuing with the registration. If the flag is still true, we loop into another wait.

The same code exists on the SendEventObject method to ensure that another thread is not sending out an event prior to processing the current thread’s event.

publicclass EventUtils {
publicstaticvoid SendEventObject(EventObject eo) {
lock (Global.LockObj) {
// Don't register a listener or process a new// event/message while an event/message is being deliveredwhile (Global.EventBeingDelivered)
Monitor.Wait(Global.LockObj); //// Cause New Listeners and events to wait until// message is delivered to all existing listeners
Global.EventBeingDelivered = true;
System.Diagnostics.Trace.WriteLine("Event processing commenced");
Global.NewEventID++;
// Add the current event to the list.
Global.EventsList.Add(eo);
// Register the amount of listeners waiting to get events
Global.ListenerCount = Global.WaitingListenerList.Count;
// Release the waiting response threadsforeach (AutoResetEvent waitingListner in Global.WaitingListenerList) {
waitingListner.Set();
}
}
// Before clearing the event and the listener// waithandles, wait for all the listeners to release.
Global.AllListenersReleased.WaitOne();
lock (Global.LockObj) {
Global.LastEventID = Global.NewEventID;
// If all logged-on users got the last event, clear the events listif (Global.loggedInUsers.Count == Global.WaitingListenerList.Count) {
// TODO: Need to clear the Global.NewEventID,// but also figure a way to notify the threads to reset their event ids
Global.EventsList.Clear();
Global.NewEventID = 0;
Global.EventsListResetId++;
}
// Clear all the listeners
Global.WaitingListenerList.Clear();
// Once an event has been delivered to all waiting listeners,// allow new listeners waiting to register to attempt registration,// or process a new event/messages.
Global.EventBeingDelivered = false;
Monitor.PulseAll(Global.LockObj);
}
System.Diagnostics.Trace.WriteLine("SendEventObject - " +
"Released listeners and cleared lists");
}

If you’re wondering why the Global.LockObj isn’t enough to synch the two methods, that’s because both methods exit the lock block and go into a wait state. The Listen method waits for an application event, and the SendEventObject waits for all Listen methods to complete before continuing to do some housekeeping.

Event Caching

As I mentioned above, we need a mechanism to cache events until we can ensure that all clients have received them. This is achieved logically, not technologically.

All events are cached in a global list of application event objects called EventList. By retaining an incremental EventId for each event, and what I call an EventsListResetID, both globally and on the event object being pushed back, I can determine if events occurred since my last Listen.

The PrepareEventForSending method returns the events to the client from the point it last received an event.

// <summary>/// Each waiting response thread will call this method
/// to return the current and all outstanding events.
/// It's assummed that all events missing by the client
/// are retained in the events list since the list
/// is not cleared until all logged on users a fully notified.
///<spanclass="code-SummaryComment"></summary></span>///<spanclass="code-SummaryComment"><paramname="fromEventIndex"></param></span>///<spanclass="code-SummaryComment"><returns></returns></span>internalstatic EventObject[] PrepareEventsForSending(int myLastEventID,
int myLastEventsListResetId) {
// If the eventlist hasnt been reset// since the last time the client received an event,// return the next event in the list.if (myLastEventsListResetId == Global.EventsListResetId) {
if (myLastEventID > Global.NewEventID)
myLastEventID = 0;
}
else// List has been reset since client last// got an event, so return all events from// the top of the list.
myLastEventID = 0;
// Determine how many events the client is missing.int numberOfEventsImMissing = Global.NewEventID - myLastEventID;
// Determine where in the list the missing events start from.int startIndex = Global.EventsList.Count - numberOfEventsImMissing;
int returnIndex = 0;
// Assemble an array of missing events for this client.
EventObject[] eventToReturn = new EventObject[numberOfEventsImMissing];
for (int missingEventIndex = startIndex;
missingEventIndex < Global.EventsList.Count;
missingEventIndex++) {
eventToReturn[returnIndex] = Global.EventsList[missingEventIndex];
eventToReturn[returnIndex].EventID = missingEventIndex + 1;
eventToReturn[returnIndex].EventsListResetID = Global.EventsListResetId;
returnIndex++;
}
return eventToReturn;

The EventsListResetId gets incremented every time the EventList gets cleared. The EventList gets cleared after all registered clients receive an event.

You’ll notice that when the client’s last event list ID doesn’t match the global one on the server, we return the entire list to the client. That’s because the mismatch indicates the client’s last EventID (which acts as an index on the list) is no longer valid because the list has been cleared since the last time the client was listening for events.

In other words, a mismatch between the client's last EventsListResetId and the server’s EventsListResetId means that the last time you got an event, everybody got the event, and we cleared the EventList. So, your EventID is no longer valid, and can now start from zero, meaning, give me the events from the beginning of the new list.

Clearing the list is done at the end of the SendEventObject method by comparing the number of logged on users to the number of registered waithandles that are blocking a listener. If both are equal, then it’s assumed that all logged-on users received the last event and any outstanding events, so we can clear the list and increment the EventsListResetID.

In the Listen web method, a block is set with this value as a time limit. When the limit is reached, the method stops blocking, and a dummy event is set to everybody using the same mechanism we used to send real events (that’s why everyone will be notified). The event class used for dummy events is the parent class to all my event classes. The client knows to ignore these events, and re-requests a Listen.

//********************************************************************// Wait for an event, but continue before the request times out.
listenTimedOut = !waithandle.WaitOne(Global.EventWaitTimeout, true);
//********************************************************************// When the code continues from here it's either because // an event was recieved or the wait for one has timed out.//********************************************************************
System.Diagnostics.Trace.WriteLine("Listen - Waithandle released");
if (listenTimedOut) {
// No events to return? Return an empty non-event.if (Global.EventsList.Count == 0) {
EventObject dummyEvent = new EventObject();
lock (Global.LockObj) {
dummyEvent.EventID = myLastEventID;
dummyEvent.EventsListResetID = myLastEventsListResetId;
}
// Fire off the dummy event asynchrously// and wait for it (short wait) so as to stay // within the wait/event paradigm in this example.
SendDummyEventDelegate sendEventDel =
new SendDummyEventDelegate(EventUtils.SendEventObject);
sendEventDel.BeginInvoke(dummyEvent, null, null);
waithandle.WaitOne();
}
}

Conclusion

With a minimal amount of code, you can create servers that serve up real-time events to your client applications. Although I haven’t tried it, I’m sure this same technique can be used for web-clients using AJAX and page methods, or at least web service methods.

Note: If you’re using the ASP.NET development server to run the sample app, you’ll need to stop it between runs.

This is very interesting, and I will give it my full attention as soon as I get some time.

However, there are two things I would like to bring to your attention.

1. Two way communication is available using WCF, but the last time I looked, it seemed so complicated that I didn't bother to learn how to do it, and rarely rely on two way communication(even though I have many uses for it).

2. Microsoft is considering putting 2 way communication into the next version of SilverLight(SL2 Beta 2). It's unknown if this feature will make the cut for the next beta, but it's likely going to be there in the final version(Chat application/P2P developers will cheer loudly).

Even so, I still feel that your code is worth taking a good look at since it addresses the simpler, and better known ASMX Webservices.

AFAIK the WCF's duplex model requires the client to establish a listener for callback connections (like in remoting in .net20), therefore it makes the usage a slightly more complex in case of internet scenarios (client behind NAT or firewall etc.). Your solution is better in this case. Question is the scalability problem. I'll check your code if I can find something.

Scalability is a known issue regarding this solution due to limited IIS threadpool threads. Each Listen for each client holds on to a thread. As I mentioned in the article this a configurable value. But users of this technique must be aware of this limitation and plan accordingly.

Yep, I know you mentioned that, maybe I just typed that wrongly - your solution have benefit of being more internet friendly than WCF duplex mode, but for the price of the scalability.

Anyway I think it can be solved by extending your solution by using asynchronous WS on the server as well (BeginListen, EndListen). This will allow you to release the IIS threads and therefore should unleash the full potential of the solution, because simply throwing more threads to the IIS threadpool is not a solution.

I'm interested in this topic, cause I'll need to design some solution for the problem in a not-so-far-future

If asynchronous WS on the server execute in a different threadpool (I believe Asynch Pages do) than what IIS uses, and the response will fire off the ListenCompleted event on the client, then scalability has been improved. Still a thread is being taken and reserved, but IIS thread configuration can be avoided.

Well unfortunatelly they are executed in the same threadpool, but you can use the following method I'll try to describe:

We'll start with BeginListen - this adds an AsyncOperation to the list, returning IAsyncResult virtually immediatelly. Thread is free to use for process another requests, only the list of waiting listeners was "increased".

When new event occurs (login, logout, new message), the list of async operations is being processed - sending appropriate message to the listening parties (required info stored in the AsyncOperation). This can also fire EndListen and the request will be completed or when the return type will be derived from the IList interface, there's a slim chance to send continuous stream of messages without the need of re-listen. But this needs further investigation from my side .