File size

File size

You first learned about Rx on C9. We've led you through the basic concepts of reactive programming to the deep mathematical foundation behind Rx (interface duality). By now,
you should understand that IObservable is the dual of IEnumerable. Today, you will learn some new concepts (for many of you) in addition to the introduction of Rx's newest interface, IQbservable, the
dual of IQueryable. In effect, the addition of IQbservable completes the interface puzzle within Rx. But what does this mean?

The great Bart De Smet takes us through the fundamentals and specifics behind this new interface, which ships in the
latest version of Rx. Most of the time is spent at the whiteboard. There's also a short demo at the end of the conversation. Here's the flow:

What operators are available (answer: 99% - explain why that 1% is omitted)

Demo:

Sample observable LINQ provider (LINQ to WQL)

Put your thinking caps on, turn up the volume, sit back, and learn. Erik Meijer and team are innovating at a level
we haven't seen in a while around here. Rx is profoundly evolving and taking LINQ along for the ride. Incredible work!

Enjoy this latest episode of Going Deep. Ask questions. Bart et al. will answer them here and on the Rx forums.

"Bart's explanation for staying in the monad in the asynchronous observable monad is nice. cancellation and composability."

Good observation ( or qbservation ) => Everything about Rx, from it's foundation to the API, is related to composable system design and development (aka software compositionality or composability). Monads
are the pieces in a truly composable software puzzle. Rx abstracts the monad from your mind and subsequently your fingers tapping the keyboard. You get to focus on the task at hand: building reliable asynchronous systems,
orchestrating concurrency, as Bart said.

Wait a minute, perhaps one need only think of monads as things that reliably encapsulate compositionality, composable pieces of computationalfabric. When you leave a monad during execution, then you leave the comfortable, reliable white sand beaches of the composable world for one rife with unexpected or random behavior.

Wonderful video, great to have Bart back on camera again. I have to agree about "IQbservable" being visually just too close to IObservable. Has "IQbservable" been run past the Framework Design Guidelines team at Microsoft? Well, the
remaining FDG team (given that some of them have gone to work at Google...). Aside from the naming issue, wonderful, deep stuff. Keep it coming!

Thanks for watching and for the feedback. We're open to all sorts of discussions on the
Rx forums, including the
naming ones. This said, we tend to try getting the semantics right first, since
alpha conversion is a widely understood technique. Not trying to downplay the importance of API design in the large,
Wadler's Law seems to capture exactly what we're talking about .

IQbservable will never make it into the base class library. That will not happen. I am absolutely sure of it. In this vid Kim goes
through some of the naming conventions the Rx team had in various API inceptions, and what the BCL changed them to after significant deliberation, introspection, cogitation, ratiocination, disambiguation and no doubt disputation.

Expression Trees on MSDN, as a good starting point to familiarize yourself with the tree APIs

Things not mentioned in the video:

Some "imperative operators" using interfaces like IConnectableObservable, requiring code blocks (and hence statement trees in a fantasy homo-iconic world), are not available on Qbservable. There are just a handful of overloads that fall in this camp such
as the parameterless Publish method (the lambda version is there though).

Qbservable also provides support for join patterns as expression trees. That is, the And and Then operators from System.Joins are available in an expression tree form as well. It's left to the reader to dream up the endless possibilities this enables for
joining multiple data streams "in the cloud".

Constructors like FromAsync which return a Func<T1, ..., Tn, IObservable<TResult>> are supported too. They return a similar Func returning an IQbservable<TResult> instead. That IQbservable will contain an expression tree with an Invoke node upon invoking
the returned function.

Parameters on Qbservable operators are only protected against null reference inputs. That is, no operator-specific enforcements are made. For example, one is allowed to write Take(-5) against an IQbservable, but not against an IObservable. It's up to the
provider to decide what -5 means (take last 5?, don't allow?).

Observable has a static get-only Provider property that exposes the IQbservableProvider that can be used to create expression trees for "local" queries and can be used to get stuff like Return in an expression form, using Observable.Provider.Return(5),
returning an IQbservable<int>. This is complementary to the AsQbservable operator.

Use of AsQbservable on a "local" observable sequence allows one to write expression trees against such a source. When calling Subscribe on the resulting IQbservable, which also implements IObservable per definition, the tree gets rewritten to target the
Observable.* operators, including join patterns and whatnot.

For signatures Qbservable operators, notice that only a minimum set of parameters has been tweaked to be expression-friendly compared to the corresponding Observable operators. That is, things like Amb(IObservable, IObservable) only get the first parameter
as IQbservable. This opens up for quite some flexibility; providers can type-check on the second argument.

Play and amaze us with great observable providers .

bryaneddsAn ​individuali​st is he who is saving himself from all those who are saving the world.

Oh, and the "visually close" argument raised earlier is rather weak, especially since homo-iconicity is all about that . Is x => x too much like ... x => x, while they may be radically different behind the scenes (delegate versus expression tree)? Also
keep in mind the interface is not what users of providers see, at all.

I also think IQbservable is pretty horrible. Even IQObservable (eye queue observable) would be a lot better. But I think the full IQueryableObservable is best, especially since I'm almost never actually going to type the name, just like IQueryable and IEnumerable
in LINQ context. The 'var' keyword takes care of most cases, and IntelliSense takes care of the rest. Consider that to write foo.AsQueryableObservable all I need to type into VS2010 is "foo.AQO", and that the clarity of the name is more important than a bit
of extra typing.

By the way, don't think that the name was the most interesting part of the vid, it's just the only part we have something to disagree on

PerfectPhase"This is not war, this is pest control!" - Dalek to Cyberman

Bridging Rx with MSMQ is totally doable. At first sight, it doesn't look like there's a domain-specific queue query language you can apply to contents of a query, so the queryable approach seems redundant. Nonetheless, some of the System.Messaging APIs may
benefit from mapping onto Rx equivalent operators. For example, the TimeSpan parameter used to specify a timeout on various Send/Receive operations is an ideal candidate for exposure through the Timeout operator of Rx.

Concerning this bridging, simply creating operators that expose a message queue as an IObservable and vice versa would likely suffice. In fact, I've done such a thing in the past. Transactions could be totally part of this too. Basically, you'd check whether
a MessageQueue object is marked as Transactional, and if so, you'd likely do something along those lines:

When importing an IObservable to MSMQ, you'd likely want a transactional send of many OnNext messages, rolling back the transaction on OnError and committing it on OnCompleted. This won't work well for infinite sequences of course (and likely you don't
want to do a transactional "send just one message" kind of thing).

When importing an MSMQ queue as an IObservable, you'd likely want transactional receive to work on a per-message basis, only committing the transaction if the OnNext message you send out to the consumers gracefully returns. That is, you guarantee the user
has seen the message successfully and could handle it.

If this automatic transaction management doesn't work for you, a better option may be to have the user control the transaction himself, simply by use of MessageQueueTransaction objects. Not as sexy or smart as the above, but maybe necessary to accommodate
for various more complicated scenarios.

I'll leave it as an exercise to the reader to implement such bridging operators.

(Edit: wanted to paste code, but it went terribly wrong; will try again later)

While Sum in IE/IQ could return an IE/IQ, that would actually (for most people) be unexpected design. Most people would rightly expect a summation operation to return a single item of the element type of the sequence (IE<T> -> T). They would not expect
it to return some sequence that just happens to contain a single element. The design of Sum in the enumerable/pull/synchronous world is correctly chosen.

The observable/push world is different in its asynchronicity, and that's the motivation for the design there.

@Paulo: I agree.. I've already complained to a couple folks on the team. I've suggested "IObservableQuery" since they don't like "IQueryableObservable".

One versus many is a popular debate amongst language people. In fact, certain functional languages that will remain unnamed encode a lot of stuff using lists (e.g. "nullability" in terms of zero-length lists to represent "not present"), while others introduce
whole separate language features to deal with 0 versus 1 cardinality (e.g. "option"). I'm much more in the former camp as
many can encode one, but not vice-versa.

Whether or not the IE/IQ design for aggregates it the right one is something that can be debated for a long time in a similar manner. If we're doing LINQ to SQL using it anyway, isn't a single result a single-column, single-row table? People have always
used different methods on low-level APIs a la SqlCommand depending on the desired outcome. While you could use ExecuteReader to get a scalar value back, ExecuteScalar makes things simpler. But if LINQ generalizes operators over sequences, something can be
said about it being closer to the ExecuteReader approach. With the sole argument of one more method call adding friction, xs.Sum().Single() isn't too bad. This said, I do buy the arguments made.

The thing that's certain is that in an asynchronous world you don't want to leave the monad. There are multiple reasons for that. One is
cancellation, but the more important one is to preservecomposition. The former is definitely an IO thing, but the latter applies equally well in the world of IE. We may introduce similar aggregates in our Interactive Extensions (Ix) assembly for LINQ to Objects as well. Simply because the original
LINQ design didn't account for further composition, doesn't mean it's not better. In particular, with operators like Amb and Timeout (amongst many others) introduced in Rx and Ix, composition for aggregates is incredibly useful. Also, a different return type
outside IE requires people to write different code to do data binding to the result of an aggregation versus a grid-shaped result.

Finally, in the IQ case, the fact you loose the expression tree breaks composition on that level in certain cases. Not if you're "lucky" and things like Sum() appear in a selector lambda expression. But if you want to translate computations involving multiple
aggregates (agreed, not often done in today's world), you're left out in the dark as there's no way to get the tree representing Sum without triggering its execution (unless you manually reconstruct it using infoof(Queryable.Sum) in a MethodCallExpression).
Btw, if aggregates stayed in IE, computation over multiple aggregates could be done easily using operators like Zip, which is just a "lifted binary operator" for single-element sequences.

The point I was making is that the final implementation should match the characteristics of what would be built with it. In Rx's case, Sum-as-sequence makes perfect case as you adequately explain. In IE's case, Sum-as-scale fits well with what people almost
certain end up doing. In any event, a user-study would reveal this (albeit too late to change the established SQO specifications if I'm proven wrong).

There are a few samples where the form quits before event subscriptions are disposed; it may be that's the case you're hitting, which looks similar in terms of exceptions. I'll have a look. Thanks for letting me know.

The sample of the LINQ to WQL provider can (finally) be downloaded. More information can be found on
this post on the Rx forum. Please use the forum for further questions and/or discussions on the subject.

Remove this comment

Remove this thread

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation,
please create a new thread in our Forums, or
Contact Us and let us know.