I work in finance, and at the moment work in the Foreign Exchange (FX) space, where we have lots of requirements that involve streaming of various things, such as:

Rates

Trades

Other financial instruments

Historically what we would have done to accomodate these requirements was to use sockets and push our own data (usually JSON or XML) down the wire, and then had some quite knarly listeners above that, where each listener would want some slightly different view of the original streaming data.

Thing is, time has never stands still in the world of software development, and new things come out all the time, literally every other day a new framework comes along that helps improve what came before it in some way or another.

One particular thing, that I think a lot of people actually misunderstood was Reactive Extensions RX, which in fairness has been around a while now. I think a lot of people think of RX as LINQ to events, which it does have. However the truth is that RX is a superb framework for creating streaming applications and the more you get into RX the more you tend to see everything as a stream, from and event, down to picking a file or a ICommand.Execute in a ViewModel say.

RX comes with many many tools in it's arsenal, such as

2 core interfaces IObservable / IObserver

Lots of LINQ like extension methods for IObservable

Time / window based operations (LINQ doesn't really have these)

Concurrency schedulers

Stream error handling controls (Catch etc etc)

Ability to create IObservable from a great many other things, such as

IEnumerable

Task

Events

IObservable Factories

I personally feel that a lot of people could make use of RX to great gain, if they were to put in the time to learn about it, a bit more.

There is also another great library from Microsoft which facilitates push based technology, namely SignalR. Quite a lot has been written about SignalR, but you rarely see that from a desktop point of view.

Most people would probably associate a SignalR Hub with a web site, but there are various ways you can use SignalR, which also make it a good choice for desktop (ie non web) development. This is possible thanks to the SignalR OWIN hosting API, such that you can self host a SignalR Hub. There is also a .NET client, we can use to communicate with the hub. SignalR also has quite a lot of events within the .NET client, and guess what we can do with those events, yep that's right we can create IObservable(s) streams out of them, which opens up the gates to use RX.

This article will show one area where RX excels, which is in the area of streaming APIs, where we will be using RX/SignalR

namespace System
{
// Summary:// Defines a provider for push-based notification.//// Type parameters:// T:// The object that provides notification information.This type parameter is// covariant. That is, you can use either the type you specified or any type// that is more derived. For more information about covariance and contravariance,// see Covariance and Contravariance in Generics.publicinterface IObservable<out T>
{
// Summary:// Notifies the provider that an observer is to receive notifications.//// Parameters:// observer:// The object that is to receive notifications.//// Returns:// A reference to an interface that allows observers to stop receiving notifications// before the provider has finished sending them.
IDisposable Subscribe(IObserver<T> observer);
}
}

Defines a provider for push-based notification.

This type parameter is covariant. That is, you can use either the type you specified or any type that is more derived. For more information about covariance and contravariance, see Covariance and Contravariance in Generics.

Subscribe Notifies the provider that an observer is to receive notifications.

namespace System
{
// Summary:// Provides a mechanism for receiving push-based notifications.//// Type parameters:// T:// The object that provides notification information.This type parameter is// contravariant. That is, you can use either the type you specified or any// type that is less derived. For more information about covariance and contravariance,// see Covariance and Contravariance in Generics.publicinterface IObserver<in T>
{
// Summary:// Notifies the observer that the provider has finished sending push-based notifications.void OnCompleted();
//// Summary:// Notifies the observer that the provider has experienced an error condition.//// Parameters:// error:// An object that provides additional information about the error.void OnError(Exception error);
//// Summary:// Provides the observer with new data.//// Parameters:// value:// The current notification information.void OnNext(T value);
}
}

The IObserver<T> and IObservable<T> interfaces provide a generalized mechanism for push-based notification, also known as the observer design pattern. The IObservable<T> interface represents the class that sends notifications (the provider); the IObserver<T> interface represents the class that receives them (the observer). T represents the class that provides the notification information.

An IObserver<T> implementation arranges to receive notifications from a provider (an IObservable<T> implementation) by passing an instance of itself to the provider's IObservable<T>.Subscribe method. This method returns an IDisposable object that can be used to unsubscribe the observer before the provider finishes sending notifications.
The IObserver<T> interface defines the following three methods that the observer must implement:

The OnNext method, which is typically called by the provider to supply the observer with new data or state information.

The OnError method, which is typically called by the provider to indicate that data is unavailable, inaccessible, or corrupted, or that the provider has experienced some other error condition.

The OnCompleted method, which is typically called by the provider to indicate that it has finished sending notifications to observers.

Anyone that has used RX a bit, will come to the point that they may like to create their own operators. In LINQ this would be done by creating a new method (perhaps an extension method) that simple returns a new IEnumerable<T>. In RX things are slightly trickier but not much trickier. In RX you would make use of Observable.Create to create your own operators, or as a factory for creating new a Observable<T>. Here is a simple example:

Observable.Create creates an observable sequence from a subscribe method implementation. You would typically use an extension method to an existing IObservable<T> and then put an Observable.Create(..) as the body of the extension method.

The interesting part of this method signature is that you can see that it returns a Func delegate hat takes a IObserver<TSource> which allows you to push values into the observer using its OnNext/OnError and OnCompleted methods. The Func delegate returns a IDisposable, which will called when disposal happens. Now RX comes with a great many different IDisposable types, don't be suprised if you see some quite exotic IDisposable types being used inside of an Observable.Create implementation.

One of the best descriptions of using Observable.Create is from Lee Campbells site :

I had the pleasure of going on a 2 day Rx course with Lee Campbell the author of like the only Rx book on the planet, which Lee has a blog dedicated to, right here : http://www.introtorx.com/

Lee now works for a consultancy called "Adaptive" who were the ones that actually ran the training course I attended. They have also released into the wild a truly great demo app which showcases how to create a full connected fault tolerant streaming API + client app. They have done several flavours of the demo:

Thing is this solution is reasonably large, and if you are new to Rx or are just trying to get your head around how to create a reliable streaming API, this solution is a tad over bearing, at least in my opinion.

To be fair I think the demo app the Adaptive crew have put together serves as more of a marketing tool for the collective minds of Adaptive, people would look at it and go, mmm these guys seem to know their onions let's get them in. However if you are a mere developer trying to get your head into that space, you may be looking for something a little smaller to start with, you know kind of baby steps.

I am hoping that is where this demo article will fit the bill, as I have been through the pain of trying to understand the WPF version of the https://github.com/AdaptiveConsulting/ReactiveTrader demo app, and will present a very much cut down version in this article for you.

A single SignalR hub (the Adative example has a few) that would push out TickerDto objects

Clients that would subscribe to a certain topic to receive push notifications from the SignalR server side hub, and push this out via RX, such that anyone interested may use the standard RX operators

The ability to simulate the server dying, and coming back to life

The ability for the connected clients to know when the server was up/down

The ability for the connection between the client and server to be resilient, in that when the server comes back up the client should be able to recover from that

The Adaptive demo does all this (and more), but I am not Lee Campbell and do not have his depth of experience with RX, though I have to say I think I am getting it now. So that is why I wrote up this small demo app

The publisher is made up of a few bits and pieces, which are described below, one thing to note is that both the client and the server make use of SignalR, so it may be an idea to read about the SignalR self hosting, here:

There is some IOC used, AutoFac is the container used. As IOC is not the main point of the article, I will not waste any more time on it. If you want to know more about the IOC/AutoFac read the code, looking into the BootStrapper class

The MainWindowViewModel is really used to facilitate sending commands to either stop/create the SignalR Hub, or to tell the TickerHubPublisher to do things. Here is the full code for the MainWindowViewModel

In order to start the publisher, the very first consideration we need to deal with is how to host the SignalR hub. This is handled by an application scope OwinStartupAttribute, which you will find in the demo app within the App class

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Data;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using Autofac;
using Microsoft.Owin;
using SignalRSelfHost.Hubs;
using SignalRSelfHost.Hubs.Ticker;
using SignalRSelfHost.IOC;
using log4net;
[assembly: OwinStartup(typeof(Startup))]
namespace SignalRSelfHost
{
publicpartialclass App : Application
{
....
....
....
....
}
}

The OwinStartupAttribute points to a class (Startup in this case) that configures the Owin SignalR self hosting. Lets have a look at that shall we.

using Microsoft.AspNet.SignalR;
using Microsoft.Owin.Cors;
using Owin;
using SignalRSelfHost.IOC;
namespace SignalRSelfHost.Hubs
{
publicclass Startup
{
publicvoid Configuration(IAppBuilder app)
{
// Branch the pipeline here for requests that start with "/signalr"
app.Map("/signalr", map =>
{
// Setup the CORS middleware to run before SignalR.// By default this will allow all origins. You can // configure the set of origins and/or http verbs by// providing a cors options with a different policy.
map.UseCors(CorsOptions.AllowAll);
var hubConfiguration = new HubConfiguration
{
Resolver = new AutofacSignalRDependencyResolver(App.Container),
};
// Run the SignalR pipeline. We're not using MapSignalR// since this branch already runs under the "/signalr"// path.
map.RunSignalR(hubConfiguration);
});
}
}
}

So that is how the hub gets started. But how does the hub know about clients, and know how to push out to them? We will look at that next

The TickerHub is the only hub in the application, and it has 2 main methods. One to allow subscriptions from clients, and one to remove a clients subscription. Here is the code for the hub, Don't worry we will be looking into these 2 methods in a bit more detail in a minute

This TickerHub method is called via the SignalR clients, and will be used to register an interest in a topic, which SignalR calls "groups". Essentially what happens is, when a client launches, it uses it own SignalR proxy to communicate with the server side self hosted hub, and will call the server side hubs SubscribeTrades() method. When that happens the clients connection is added to the server side hubs group, and the client is sent a one of push of all currently held tickers. This single push of all the current tickers, is done by pushing the entire contents of ALL the TickerDto objects currently stored in the TickerRepository, which is a simple in memory queued repository of the most recent ticker TickerDtos produced.

Any time a subsequent ticker is created (we will see how this happens soon) the client will get that pushed at them in real time. This bulk push is a one of, when the client 1st establishes comms with the server. The other thing that happens right at the start of this method is that the context is captured such that external classes from the TickerHub, will have access to the TickerHub context and its associated values

This TickerHubUnsubscribeTrades() method is called via the SignalR clients, and will be used to un-register the clients interest in a topic (which SignalR calls "groups". Essentially what happens is, when a client shuts down it uses it own SignalR proxy to communicate with the server side self hosted hub, and will call the server side hubs UnsubscribeTrades() method. When that happens the clients connection is removed from the server side hubs group.

Ok so we have now seen how SignalR clients register/unregister an interest in a topic (at least from a server side perspective), but how does the client get push notifications from the TickerHub? Who does that? Well that job falls to another component. Namely the TickerHubPublisher. The job of the TickerHubPublisher in this demo app is quite simple, produce a random TickerDto and push it to any connected clients, who are interested in a particular topic. The entire contents of the TickerHubPublisher is shown below. I think the code is pretty self explanatory.

This method makes use the TickerHub context that was captured within the TickerHub itself. By capturing the context, it allows external classes from the TickerHub to make use of its properties/values. That is precisely what the TickerHubPublisher does. It simply uses the captured TickerHub context to push out new values to subscribed clients for a particular topic. Which in this case is the value of the TickerHub.TickerGroupName ("AllTickers") string. So that is how the SignalR server is able to push notifications to the clients

The client is a standard WPF app (I am using WPF just because I know it, it is not really that important, you could use Winforms using the MVP pattern just as well). The client also makes use of a SignalR proxy class, and as such a good starting place would be to examine the SignalR .NET client APIs, which you can read more about here:

There is some IOC used, AutoFac is the container used. As IOC is not the main point of the article, I will not waste any more time on it. If you want to know more about the IOC/AutoFac read the code, looking into the BootStrapper class

There is an awful lot going on in this class. As I say THIS class is a core class. So what exactly does it do?

Well it does a few things

First and foremost it provides an IObservable<T> based on the factory which produces the final IObservable<T> that will be yielded. This factory will be the unique source stream (IObservable<T>) that needs to be made resilient.

It uses a connection status stream (which internally makes use of the SignalR events (such as connected/disconnected) to create the status stream) to merge into the overall stream. When a "Reconnection" or "Closed" status stream result (OnNext value) is seen, we consider that to be a disconnection, and we synthesize an OnError notifcation (by using the standard Rx Dematerialize operator) that is merged with the actual facory streams value, to give an overall stream result, which could potentially now yield OnNext/OnError to its consumers

We also use the standard RX Publish operator, we do this to share the same underlying IObservable<T> sequence

We also use the standard RX RefCount operator, such that when there is no subscribers the sequence will be disposed

This class makes use of a couple of extra uiltity classes, which we shall look at next

The Connection itself exposes a SignalR proxy and also a IObservable<ConnectInfo> which is creating by examining all the available SignalR .NET client events. Consumers of this class will now have acess to a stream for the connections, such that they may be able to find out when a connection is lost or reconnected.

As we saw in the publisher, we know that a client to a particular server side SignalR hub will call 2 things, namely SubscribeTickers and UnsubscribeTickers. And we also know there is a streaming API of TickerDto (At least there is for this demo app, which only contains a single hub). Where the following is true:

The client will call the server side hub SubscribeTickers on connection

The client will call the server side hub UnsubscribeTickers on disposal (this is done by only calling this method on disposal, see the following code where we use the power of Observable.Create(..) along with some CompositeDisposable to dispose of a few things, one of which is an Action which we create using Disposable.Create(..) where that Action will call the UnsubscribeTickers on disposal.

The other thing to note about this code, is that it inherits from ServiceClientBase, and as such has the resilient connection functionality that we discussed above. Where we feed into the ServiceClientBase class the GetTradesForConnection(IHubProxy tickerHubProxy) method to act as the IObservable<T> factory that is used in the final (now resilient thanks to the logic in the ServiceClientBase base class) GetTickerStream stream.

From here the GetTickerStream stream is free for anyone to use. So lets see who uses it, that is discussed just below.

We use Observable.Defer such that we do not actually make use of the underlying stream, until someone subscribes to the IObservable created by using Observable.Defer. It is a way of making a hot stream cold.

We use select to transform the strem data from TickerDto to Ticker

We use Catch to catch any Exceptions (OnError) in the stream. Where we use a default value for that case

We use Repeat. Now this one is VERY VERY important. This is the one that allows us to repeat the whole stream, including connecting to the server side hub again. It is this mechanism that allows the client to recover from a server side loss of SignalR hub. This along with the resilient stream logic are the MOST important bits to the app (at least in my opinion)

We use Publish to share the underlying stream

We use RefCount to get automatic disposal when there are no longer an subscribers

So now that we have seen this repository, we only have one more hop to continue the IObservable journey for the TickerStream. Let see that next.

The TickersViewModel represents ALL the Ticker(s) you see on the screen. It is this viewmodel that makes use of the lazy / repeatable / resilient IObservable that the TickerRepository provides. Lets see the code it is pretty self explanatory, I think:

Where each Ticker is represented by a single TickerViewModel which is as shown below. It can be seen that this class also makes use of the ConnectionStatusStream IObservable we discussed earlier. This is used to change the TickerViewModels view to show a red box with "DISCONNECTED" in it. We will talk through that in just a minute.

The last thing I wanted to show you was how the ConnectionStatusStream was used. This stream OnNexts when the client side SignalR hub proxy events are raised. So we see thing like Connecting, Connected,ConnectionSlow, Reconnecting,Reconnected,Closed,Uninitialized. All of which started out life as events on the SignalR hub proxy, and are turned in to an IObservable stream using one of the many RX factories. In this case IObservable.FromEvent.

Anyway here is the overall ConnectivityStatusViewModel that we use to show the information in the bottom status bar of the app.

Well there we go, I hope I have shown you all that RX is actually way more than just LINQ to events, it contains a great many classes for dealing with concurrency, LINQ, events, streams, factories, retries, error handling, disposing of resources.

Some of you may be pleased to know this is a 2 part article, in the next part I will be teaming up with the author of the .NET port of the highly regarded socket library ZeroMQ. The .NET port is called NetMQ, and together with its author (Ok I did contribute a bit to that library too) we will be showing you how you can build this sort of app using NetMQ. So until then, if you have enjoyed this article, please feel free to leave a comment/vote they are most welcome

Comments and Discussions

Hi Sacha,
thanks a lot for your exhaustive and comprehensive intro to Rx/signalR. I'm wondering how to enhance your while-loop by using a FileSystemWatcher when it comes to real stock ticker integration. I've implemented a simple reader / Scanner for realtime push stock tick notifications that scans a financial web site 4 times a second and serializes each stock tick change to a file eg.: MSFT-BID-45-15-ASK-43-67-TIC-2017-10-31-13-06-14.TXT. This works fine and reliable. Several thousand files are generated during a trading day.

But I stuck to connect my producing bits with your consuming bits using a FileSystemWatcher detecting new Ticks (File created) establishing a poor man's version of an DIY event stream.

Sorry to disturb you. i have few question about signalr to you because you are very advanced developer what i deduced from your article depth.

1) suppose if some one need to develop a web based chat apps where million of user will login and chat in each other then signalr would be good choice ?

2) i have seen people suggest to use socket.io and other few library to develop web based any kind of communication library and they avoid signalr. so tell me what is your suggestion about signalr ?

3) did you use signalr in big project where load will be higher ?
4) signalr allow peer to peer communication ?
5)how to transfer file peer to peer by signalr ? if this is not possible then which library exist to do this efficiently which work across all the major browser ?

6)i could not found any article link which show how we can use signalr to store connection detail in sql server instead of in memory.......share a good link for this if you know any article written by you or other.

7)one problem i saw that signalr connection id change when client reconnect. so how to deal this situation.

Hope all are well since the last time you helped me. Thanks anyway for that!
I only saw now that it is again on 15 Feb that I am asking for assistance,

This time around I have an issue with socket leaks it seems. My solution has been running in production for the past year, with only minor hickups that I sorted out. However I found now that there are actually major issues with TCP connections that stays open, even though, if I am not mistaken, that these connections should be closed and cleared whenever the connection and stream is disposed.

I have again gone through my implementation, which is pretty much still what you have in your article and what you suggested previously. I just can't find where or why the TCP connection would not be closed.

As per my previous question last year, remember that I handle additional connectivity outside the stream as well, letting the client connect when needed. Once the connection is done, the resilient stream as per your implementation will insure the connection stays open while the responses are outstanding. Then when all responses has been received and there was no activity for a few minutes, a logout is called.

My piece of the code that does the login and logout is normal implementation as can be seen here:

As you can see there is nothing funny, the logout is triggered through a timer which tracks inactivity. This time is also triggered when a disconnection error is received from the streams. Then objects are disposed. Which should close TCP connections, right?

Do you perhaps have any insight in why this could happen?

UPDATE:
After much searching and testing I couldn't find a viable solution to the issue.
To decrease the number of connections being opened I could increase the time between reconnect retries, but this would cause critical messages being lost.
The solution I have is to move the whole mechanism over to use queues, implementing RabbitMQ and still having a reconnect mechanism for the bad network in place. This seems to be working very nicely as messages aren't getting lost even if connectivity is struggling.
Side Note: Thanks anyway for the brilliant post on SignalR, for the time I have used it it did the job very nicely!!

When SignalR connects, it in essense opens 3 TCP connections for negotiate, connect and start. 2 of which is eventually closed if the connection is successful and "stable". However, on one of our production servers (with a couple of 100 client services), and where there exists slow network and disconnections, this causes major issues.

Let me put it this way, every time slow network is picked up by SignalR client, the connection is killed and reopened, since no messages can be sent over the slow connection. If you count this up on every connection retry every couple of seconds, times a couple of 100 clients all doing the same, you could easily see how TCP ports gets exhausted, and fast!... Rate of reconnection is faster than rate of TCP ports closing. Note: closing and disposing of classes, objects and connections are done and works fine. The TCP connections however still linger...

i spent almost 2 months trying to understand and make a similar app as ReactiveTrader. your post is exactly to the point, it shows a simplified version on ReactiveTrader, exactly what i missed. I wish I stumbled upon your post earlier

First I'd like to thank you for this project as it helped me out with quite a couple of issues I was having! I've been following your articles and "using" your samples since you posted about Cinch back in the day. thanks bud!!

To get to the current issue at hand, I found this article December 2015 when I had to look into pushing messages back from my API to the connected .NET client processes via SignalR. Using Rx is still a bit new to me, but I gave myself a crash course since then and what you have done here is pretty awesome.

Everything is working well, for the most part anyway. Running the services on local dev machine works perfectly. Dropped connections gets reconnected, status is kept on the occurrences and communication over the API and SignalR responses continues as expected. Deploying services etc to the Dev server for testers also works perfectly as expected.

However, once I get to UAT server deployment, things start going wrong in keeping the SignalR connection open. For the first day or so everything runs as expected. Then at some point the SignalR connection seems to "drop", at which point I can trace in my logs that the connection is recreating non stop. Log entries are milli Specifically the log messages from

These two messages fills up my logs. Only after restarting the process does the connection open properly again.

Do you perhaps have any insight in to why this would happen. As I said, both on my machine as well as the dev server for testing it runs smoothly. Services has run non stop on Dev for the pass 3 weeks without any problems.

Entries extracted from log file (all the log files for the day has only this in it.)

Just some additional information, I am managing logging in and out of my API and SignalR outside of this approach of using Rx. The reason behind this is because I do not always need to be "listening" for messages being sent from the SignalR hub to the .NET client process. Only when the client has sent a "command message" over the WebAPI, the connection to the hub is opened to wait for a response, which can take anything between a couple of seconds to about 10 minutes.
If no "command messages" has been sent to the server side WebApi for a period of 5+ minutes, then the connection is closed (the client logs out of the WebApi and SignalR) I used your approach with SignalR and Rx as per your article to ensure that the connection stays open during the time the client is awaiting the response for the sent command.

Any assistance or insight with regard to the issue will be greatly appreciated.

Apologies, saw your response just now. Will wrap the code as you suggested. Will report back with regard to the results, it just takes a long time before it actually starts with the issue. Yesterday the issue didn't come up and today so far so good.
Once it comes up again I will run the "wrapped" code and then probably only be able to report back a day or 2 afterwards.

Hi Sacha, I am trying to adapt the code to query (depending on user input) the database (and poll to check any changes to send back to the client). As the server side publisher Start method needs criteria I am not invoking the Start on startup of the server-side app (eg. within the MainWindowViewModel) - I am invoking the Start Publisher method within the Subscribe method of the hub, and the group name is whatever the search criteria is. So the Hub Subscriber code looks like

However I have been having a few problems with the above. Firstly I'm unsure how to handle the unsubscribe part (note I'm Dispose'ing the client-side GetMyDataStream object which invokes the unsubscribe for that instance of the view model before subscribing to the new) - when I search I need to subscribe and use the (new) search criteria (which is the group name) but at the same time I unsubscribe to the old group name (which was the old search criteria). The unsubscribe takes too long sometimes (especially when the user sends search criteria in quick succession - they queue a often times-out).

Secondly when I have two (or more) [WPF] controls that call the Subscribe, the message sent back to the client, actually sends back to all instances of the controls that have previously subscribed (whether they have subscribed with different search criteria or not). It seems to be sending it back to the user rather than the control instance that initially subscribed

Can you provide some insight into what might be wrong and/or the way search querying is done via SignalR - I would prefer not to use SqlDependency as this results in other issues (involving multiple/simultaneous requests, the SQL query has limitations, marking those records that have actually changed, etc)

I think I would do things differently. For a search based thing I would maybe not use SignalR to broadcast that out, as this seems very specific to a given search. I think I would make a call to say a WCF service which returned Task<t> and then create Observablee.FromAsync(callToWCFHere), and then I would broadcast that out internally using a simple Mediator / Event Aggregator type thing.

I am assuming all this searching happens in the context of one Process, or is it across processes?

Hi Sacha, many thanks for the reply. You are right in saying there will be very specific searches - it is acct#, name, etc for querying a full text catalog. The search criteria will be entered into a number of WPF user controls (so multiple grids [different instances of the same user control as well as different UC's] with async calls to server-side FTC queries) - and there are multiple desktops.

I can see what you're saying regarding an event driven Observable async WCF - that might make a more sense because it is thinner and should still update the client if there are changes to the database.

You've been a great help, but would that still use RX and do you have an example by any chance ?

The desktops (i.e. clients) won't have a need to communicate to each other - generally the search will be a read from a database query (no write back - and the database resides server-side on the same infrastructure as the hub(s)).

I would have a WebApi controller that returns the data, and accepts the query objects

I the UI I would write a very generic wrapper around calling the WebApi, say using RESTSharp/HttpClient (whatever helps you really). I would make a generic method, that returns Task<t>, and would accept generic params for your queries.

From that I would write a specific UI service for the work I want to do, which calls the RESTSharp/HttpClient code, which would return Task<t>, I would then use Obserable.FromAsync(....) which takes the Task, and gives back/returns an IObservable. From there on out I would do normal Rx code.

Many thanks for that Sacha - this seems to be a bit lighter and a better fit for the search/query than SignalR - I'll have a look at the REST/Client examples and start off simple with a REST service returning a Task called via RX, then make it more general as I go. Really appreciate your perspective - I'll get back to you as things develop

Hi Sacha, apols, just quickly - the articles look good - I just want to clarify something that doesn't seem to be implemented within the approaches. After the database is queried with the search criteria and the initial results sent back to the client, I want to poll the database to detect any change in the results - is that initiated from the client (i.e. a pulse [every say, minute] to tell the WebApi service to go and check) or does the WebApi service loop after being initially called (then it sends down changes to the client as/when they occur). If it is the latter I'm not sure how this is implemented (is it done via sending a message from the WebApi service and the RX Subscribe will take care of it?)

Ah I did not realize that you wanted to notify client of changes in the database.

You can see why this is not a good fir for SignalR, you would create loads of custom queries, which would eat a connection / group for each query. So that is not great fit. Signalr is more for push notifications of data the way I see it. So streaming data (say Fx rates) would be great fit.

For your queries I think WebApi is the right way to go. Request the data, WebApi gives you data, you create Observable on the client side which does the call to get the data.

For the database changes notifying the client you have several options

Polling (in a loop) and telling clients to update on regular intervals

SqlDependencyCache and tell client (this could be SignalR, where there would be one group something like "databaseChanged" that the client subscribes to). You would use SqlDependencyCache to push notification to clients, who would know DB had chnaged, and would do re-query to WebApi to grab the data

MS SQL Server has a service boker (which I have never used) which I think may be able to do the job.

Though this really sounds like a job for a distributed cache such as Redis. When you put something into the cache tell clients and they can use that. If they get a cache miss they can get it from DB and put it into cache.

Ok thanks Sacha - having played about SqlDependency before I'll leave that alone (brilliant concept, but never quite works as expected). I'll go for the WCF with callback with an RX client call - as I've yet to see any real examples on how to callback on WebApi. Much appreciated for the recommendations - it's been really helpful to talk thru the possibilities

Yeah WebApi is request/response only. Duplex WCF will serve you well, just hang on client callback channel. Make sure you WCF service is not percall, that would not work if you plan to callback client from WCF. You need some state saved (ie the client calbacks). I have a very good (even if I do say so myself) example of Duplex WCF : WCF / WPF Chat Application

458 votes and 1.2 million reads can't be wrong...Ha Ha

SignalR would work, but your clients would have to subscribe to a more general notification something like "DataBaseChanged". And for the queries, yeah WCF, or WebApi for sure.

The article was helpful (458 votes weren't wrong !) - the callback is in place (PerSession - which saves state for each source WPF user control instance which is perfect - thanks for the tip). For the state save I've got a Dictionary backing store to compare whether the data has changed. Huge thanks - with all the examples out there, there just wasn't that many scenarios that had the kind of search/query/polling functionality (all seemed to be not much more than snippets involving SignalR/SqlDependency).

Hello again Sacha - I'm extending what you have here into to accept inputs from multiple sources (market feeds, news, etc). I have seen this is done in the Reactive Trader solution, but it's difficult to put the different components into a context. I have a few points that I've been trying to figure out - it would be really be helpful if you could provide your thoughts

Within the HubClient classes you have a TimeSpan.FromSeconds parameter which is the connectionTimeout. Added to this you have the server-side Publisher class which sends down a ticker record iteratively (which simulates for example if I need to poll a SQL table and send any new records iteratively - say new news articles every minute). I'm a little unclear about the use of the client connectionTimeout value and whether it needs to be sync'd up with the server-side

With regards to the instantiation of the Ticker Publisher class - this seems to rely on the declaration in the server-side GUI ViewModel - if I add other Hubs/Publishers, would you advise to add the Publisher of these to the GUI ViewModel in the same way or would they 'fit' somewhere else

Both the client-side and server-side IOC create singleton objects - is this to simplify the solution (removing the need for significantly more thread logic) - i.e. would you leave as singletons or would you would remove the restriction and add thread catering logic