Setting the scene......Where I work is a Fx (Foreign Exchange) company and we
trade Fx for clients all over the world, and the other day my boss came up to me and stated that
he would like to be able to visualise where trades where they were happening in real time, but he
wanted it too look cool, a kind of shiny showcase type of thing (I am sure you
know what I mean). He categorically stated no grids. I was pleased.

My team mate Richard and I were tasked with this, so we thought about this and looked at what sort of
information we had available, and wondered if we could make
some sort of generic real time event watcher that would also produce some
sparkly interface for us to show off.

We did NOT have much information to work with, we pretty much ONLY had the
following

Tcpip address

An arbitary string that described the real time event type, for example
"ClientDeal", "ExchangeFund" etc etc

The name of the client

So we thought about it a bit more, and what we came up with was something
along the lines of this scenario.

We could extend our logging framework (we use
Log4Net),
where we could create a custom MessageQueue (MSMQ) appender, which would log
certain events and some extra data (such as Tcpip address) to a

MessageQueue

.
Obviously we could not share our entire application so we have provider a
test message publisher that simply writes test messages to a

MessageQueue

.
This part should be pretty easy to figure out should you want to come up
with your own stuff to generate the real time events.

We could have a WCF service read these MessageQueue entries in real
time. This service could take in subscribers, where each subscriber could
subscribe for a single event using the event name, or multiple events, by
passing an array of event names on subscription

We could make the WCF service use callbacks to the subscribers to push
notifications back to the subscribers in real time

We could also use Google Earth to show these events (if we could obtain
GeoLocation data for the event) in real time

Most of this is fairly standard stuff, the actual interesting part is pushing
notifications from some server side web code back to the browser in real time. I
don't know how many of you have seen that before, but it is like what Google do
when you open a Google search page and search for something that is quite
populaar (say some news worthy item), and Google will actual stream live results
at you straight into your open search page.

It's very cool, and is usually accomplished using long polling or various
other techniques using Ajax/Comet techniques, all of which are very hard to set
up and get working (at least in our opinion).

What we ended up doing was using a rather new library called
SignalR, which
I have say is pretty darned cool.

We will be going through all of the different moving parts of how we went
about this, later in the article. One thing to bear in mind is that for our
requirement we wanted to
show stuff on Google Earth,
which you may not want to do. However the use of
SignalR could
be applied to any scenario where you want to stream live data straight to users
browsers, something like search results, some sort of streaming changing data,
such as live market rates, or strangely enough FX rates. Funny that.

One thing to note is that the test publisher is picking randomly from a small
set of known TcpIp addresses, so you may see the same TcpIp (therefor
GeoLocation) picked after each other. That is the nature of randomness over a
small dataset.

Anyway click the image below to download the video (its about 180Mb sorry, screen capture software produces big files), it should be pretty clear
what is happening, essentially the test publisher project is publishing event to
an MessageQueue and the these messages are being pushed to Google Earth
in real time (streamed to it) via WCF and the use of the
SignalR
library.

There are a few prerequisites, however most of them are included as
part of the attached demo code. The table below shows them all and tells you
whether they are included in the attached demo code, or whether you really
MUST
have them in order to run the code

Item

Included

MSMQ

NO

You MUST have this turned on and running (it is a standard Windows component)

Named MSMQ Private Queue

NO

You MUST have a queue named "eventbroker" (or whatever you configure in the TestMessagePublisher and WebUI project config files)

I think the best way to get started is to consider the following diagram
which tries to outline the different parts of the attached demo code:

Each of the black outlined boxes represents a project within the demo
project, whilst the orange outlined box represents functionality that is exposed
by the use of the
SignalR dll.

We will be going into these projects and the use of
SignalR in more
detail below, but for now here is a very short description of what each of these
projects does.

Codeproject.EventBroker.TestMessagePublisher: This is throw away code. The sole job of this project is
to simulate the generation of real time messages

Codeproject.EventBroker.Service: This is a duplex WCF
service that reads messages from a MessageQueue that the

TestMessagePublisher

is writing to. To run this service you will need
to start the WCF host project Codeproject.EventBroker.Host

Codeproject.EventBroker.WebUI: This is stndard ASP .NET
project which host an instance of Google Earth
in a web page. The web page also calls some server side
SignalR code
which then subscribes to the WCF service and will also accept callbacks
which provide real time values which are then shown in real time on Google Earth
using
SignalR.

It can be seen that the default queue name is "eventbroker" which is expected to be a private queue on the local machine.
But this queue could be anywhere, this is just to show you where you can
configure this.

One other important note is that the "eventbroker" MessageQueue
queue MUST NOT be transactional. As the demo code assumes it is
not transactional, if you want to make the queue transactional you will need to
alter the Codeproject.EventBroker.TestMessagePublisher MessageQueue
writing code and the Codeproject.EventBroker.Service MessageQueue
reading code.

If you want to make it transactional that's fine, but you WILL have to change
code to do that. Also be aware that you will have to give access rights to
a user to the newly created "eventbroker" MessageQueue queue. I
tend to go with my own login and grant all rights.

For each event name that the subscribers wishes to subscribe for do the
following:

If there is currently no subscriptions for the that event name,
created an empty subscription list

See if there is already a subscription list for that event name, if
there is add the subscribers Id and callback context (IEventBrokerCallback)
to the global dictionary of subscribers for the event name

Once a sunscriber chooses to end their subscription they may do so using the void EndSubscription(Guid subscriptionId) operation contract.

When subscriber ends a subscription the following occurs

For each event name in the global dictionary of subscribers for the event name

Get all subcriptions that do NOT have the same subscriptionId as the
subscriber that is unsubscribing

Create a new global dictionary of subscribers of those subscriptions
that remain after removing all subscriptions that are no longer need due
to a subscriber ending a subscription

Here is the most relevant code when a EndSubscription occurring:

publicvoid EndSubscription(Guid subscriptionId)
{
lock (syncObj)
{
//create new dictionary that will be populated by those remaining
Dictionary<string, List<UniqueCallbackHandle>> remainingEventNameToCallbackLookups =
new Dictionary<string, List<UniqueCallbackHandle>>();
foreach (KeyValuePair<string,List<UniqueCallbackHandle>> kvp in eventNameToCallbackLookups)
{
//get all the remaining subscribers whos session id is not the same as the one we wish to remove
List<UniqueCallbackHandle> remainingMessageSubscriptions =
kvp.Value.Where(x => x.CallbackSessionId != subscriptionId).ToList();
if (remainingMessageSubscriptions.Any())
{
remainingEventNameToCallbackLookups.Add(kvp.Key, remainingMessageSubscriptions);
}
}
//now left with only the subscribers that are subscribed
eventNameToCallbackLookups = remainingEventNameToCallbackLookups;
}
}

The interesting part of the WCF service is the actual callback to the
subscribers. Bu when should this callback occur?

Well the callback to a subscriber should only occur when we have something to
deliver to them, which is when we have an incoming message from the

MessageQueue

and it matches a subscribers subscription desires (basically
the incoming message EventName matches the subscribers

EventName

that they used when subscribing.

As this WCF service is intended to be used by many clients, there are several
threads running, there is the main WCF thread, and there is also a new Thread
spun up to handle reading from the MessageQueue and dispatching
messages back to subscribers should the imcoming event EventName
have active subscribers.

The only other clever thing this code does, is if we get a CommunicationObjectAbortedException whilst trying to
send a message to a subscriber, that subscriber is assumed to be faulted and removed. You will see that the subscriber also has mechanisms
for dealing with a faulted channel, which is not so easy when it comes to publish/subscribe. For example one subscriber could be faulted but all
others could be fine, so should we restart the ServiceHost, probably not. That is the appraoch we have taken here
we try to be as fault tolerant as possible, and only resort to restarting the ServiceHost if the channel faults completely.

Where this xml parsing code also make use of another bit of utility code that
obtains GeoLocation information from a TcpIp address. This service is free but
occassionally misses some TcpIp addresses. At work we actually use a web service
provided by MindMap, which is very reliable and costs $20 for 10,000 lookups and
is a simple GET http request. However for this articles demo code we have
provided you with the free slightly unreliable version sorry abou that.

That said the TestMessageQueuePublisher is always picking random
TcpIp addresses that we know work with the free GeoLocation lookup service
that this demo code uses, so you should be fine.

The last peice to this puzzle is a simple web site that is used to display
real time (or as near as damn it, there is a slight latency dealy go through the
various layers, in fact these are the layers a real time event goes through,
just so you can see where the web site fits it

Anyway that said the web site is pretty simply the only slightly exotic thing
about it is that it uses this fairly new free library called
SignalR which
we discuss in greater details below. In essence what the web site does is host
an instance of the Google Earth
plugin in a standard HTML page which gets manipulated by a bit of jQuery
Javascript. It also makes use of the
SignalR library
to allow push notifications to the browser.

Essentially what
SignalR is, is
a Async signaling library for ASP.NET to help build real-time, multi-user
interactive web applications. It does this in a very clever way though. It
basically allows you to write server side code that inherits from a
SignalR Hub
object. You can also then create Javascript that will communicate with the
server side
Hub object, and vice versa.

Yeah that's correct we can write to a Javascript
method via server side code, it is actually quite nuts.

Of course there is some magic, but once you fathom what is going on, it is
not that magical rather plain clever. Here is what happens under the covers

SignalR will
create a lightweight Javascript proxy that allows Javascript to talk to the
server side code

SignalR will
also create dynamic “Clients” and “Caller” objects in your hub, so that you
can invoke a client side method written in Javascript directly via your code
in the server side

SignalR will
examine your browser agents capabilities, and will do one of the following

Will 1st attempt to use WebSockets to allow Javascript
SignalR proxy
to talk to server side code

Failing the availability of WebSockets
SignalR will
revert to using long polling to allow Javascript
SignalR proxy
to talk to server side code

The JavaScript comms to the custom
SignalR Hub is
where the rest of the magic happens, but before we look at that, lets see what
you need to do on a hosting page, HTML page in our case (could be
ASP/ASPX/CSHTML etc etc)

It can be seen that we have the following script tags on the
SignalR enabled
page

Now if you look at the demo projects folders, you will NOT see a signalr/hubs folder. That is magic, and you
MUST just accept this
and know that
SignalR will be
putting stuff there. Granted a certain element of trust is required.

So once you accept that there are unicorns/pixies and elfs in coding, we can
now concentrate on reality which is how do we get JavaScript to talk to a
SignalR Hub. In
the demo code if you examine the file "GeoLocationView.js" you will see the following
section of JavaScript code that is responsible for initiating the communications
with the
SignalR Hub.

This is the really interesting part of this solution in our opinion, what
happens is that when the duplex WCF service calls the EventTicker
using the InstanceContext that the sunscriber provided, is that by
using the
SignalR Hub we
are able to directly call into a Javascript method

Here is the relevant Codeproject.EventBroker.WebUI server side
code (see EventTicker ), which is what the duplex WCF callback
calls via the initial InstanceContext that was provided by the
subscriber:

Pay special attention to the JavaScript function name, and see how the server
side the SignalR Hub
code is able to just call that, and pass across .NET objects which are then
recieved by the client side JavaScript as JSON, that is quite mental we think.
Quite mental indeed, Kudos to the SignalR
chaps/chapesses.

As we have said before
SignalR is
clever enough to detect your browsers capabilities and will try the following

One problem with do publish/subscribe is that the channel could fault for a
particular subscriber and that channel for that subscriber and its callback is
pretty much useless, but the subscriber has no way of knowing this. So how do we
combat that.

Well if we look in the Web.Config of the Codeproject.EventBroker.WebUI
project you will see the following WCF configuration

Where we see 2 timeout values Send and Receive, which are both set to 10
mins. So the approach we took was this, grab the Receive timeout value from the
WCF binding, and the start a timer, when that timer expires we do an automatic
unsubscibe to the WCF duplex service and then subscribe again. With this
approach is place we only loose a maxium set of data for the Receive timeout
value should the subscribers channel be faulted.

The Google Earth
integration is all pretty standard stuff that you can learn by reading the
various Google Earth
documentation/API reference pages. However for completeness here is what the
code looks like for Google Earth
integration.

As we say this is all very standard stuff is you use the Google Earth
API, essentially what we do is use the following code in the

Anyway that is all we really wanted to say this time. Richard and I will probably have some more stuff for you at some stage, but I am now going to get
back to the final 5% of this OSS project that Pete O'Hanlon and I are working
on. Thing is that final 5% is the hardest part, but we are both into it, so
expect to see it appearing here some time soon. Its been a while coming but we
both like it, and feel it will be of use. So until then.....Dum dum dum.

However if you liked this article, and can be bothered to give a comment/vote
they are always appreciated. Thanks for reading. Cheerio

Yea sorry... I haven't even seen it. We've talked about it and I thought it was a coincedence that we had just recently talked about it and then I read this article. He started writing a mock up for his job and I think he's working on a commercial version to sell soon. I'll keep you posted.