Description

Josh Phillips(PM), Niklas Gustafsson(Architect), and Artur Laksberg(Developer) of the Parallel Computing Platform Team spend some time with me to discuss a managed (.NET-based) DSL (Domain Specific Language) for concurrent programming, Maestro. Maestro
incorporates well-entrenched language patterns (imperative, OO, C style syntax, etc) and language constructs (channels, agents, domains) in a compelling way to make concurrent composition more accessible and familiar to the legions of sequential code composers.

Here we dig into the architecture and design of the Maestro language and compiler as well as discuss the philosophy behind this incubation project (at this point in time there are no plans to release Maestro as a product - it's a research project, an incubation...).
Why create another language to help solve the Concurrency Problem? What's the advantage over implementing a library (this is .NET after all -> CLR + BCL = most of the power of the platform)? There's obviously good reasons for implementig Maestro as a language,
but you'll need to watch and listen to find out.

Enjoy.

Note: After this interview was conducted and posted to Channel 9, the Maestro team has renamed their technology to
Axum. So, they are now the Axum team and the managed DSL for concurrent programming they're incubating is called Axum.

Any application with concurrent tendencies It's a DSL, but the specific domain is concurrency/parallelism and what it offers (at this point theoretically since you can't play with it yet) is a readily comprehensible way (providing sequential semantics
that you're most likely very used to) to program multiple asychnronous domains that communicate via channels and the agents they possess. At least, this is my understanding after listening to the big brains explain their thinking.

In the new year, this will be fixed (most likely with the addition of a low res MP4). Right now, MP4's are being encoded at a high bit rate (maybe too high). Sampy (Mike Sampson), the inventor of Magic Folder (the application that does all of the processing
magic for our media files) is aware of this and has plans to address this.

I have thoroughly enjoyed all the M and Parallelism talks. For parallelism there is enough to keep you busy for quite some time.

So now we know that Maestro is built on CCR.

CLR

TPL

CCR

Maestro

?

C/C++

CRT

PPL

?

I would be interested in

Native vs managed integration? - CRT/PPL vs TPL/threadpool

TPL/CCR integration?

What scale do you expect what libraries to be able to manage?

What level of concurrency do what libraries efficiently enable?

How do what concurrency paradigms affect program complexity and predictability and how future proof will they be

In the past sequential programs became faster with faster processors; we cannot just expect parallel programs to continue to scale - exposing concurrency at some level may seem fine, but it will start to hurt at some level -- program flow prevents absolute
parallelism; maximization how? - Niklas talks about speculative parallelism...

Dogfooding -

Where is Microsoft be dogfooding these libraries - applications and services; what is being done in Windows itself, going forward?

What will it dogfood where and why?

How deep into the OS will it dogfood and what will be used where?

What does concurrency mean for Windows?

Implicit concurrency

Is it feasible?

What languages?

What timescale?

On what scale?

Dynamic program optimization for needed parallelism (no. of cores) or static partitioning with "fixed-grained" tasks?

Parallelism concepts, how many are there both research-stage and applied

It's interesting that the
CCR is the machinery behind all this. It's an excellent piece of work and a shame (to my mind) that its adoption does not seem to have been wider. I think the actor model is certainly one of the better models for taming concurrency, and the CCR provides
a strong foundation for it, amongst other things.

I think anything that Microsoft do to promote this model is going to be a good thing in the long run, and I look forward to a possible future release of Maestro in some form.

The deep isolation present in
Erlang, which supports the actor model as a first class concept, is also available in
Decentralized Software Services, which builds atop of the CCR and is available to C# developers now.

For those interested in the functional perspective, there is a non-blocking asynchronous message passing implementation available within the F# CTP and I've been
toying with using the CCR from within F# computation expressions to simplify the syntax somewhat.

I don't know how wide the adoption has been of course. I did say that it *seemed* not to have been wide and of course I could be wrong - wouldn't be the first time - but using the only benchmarks I have, namely (1) the number of articles in the blogosphere
on CCR and (2) the amount of activity on the CCR/DSS forums, it's not an entirely invalid assumption.

Of course it could be that not many think the CCR is worth blogging about, or that it is such an obviously intuitive model that not many need the forums, but I think neither of these is true and your own comments in the video suggest that some developers may
consider the programming model something of a mental leap from where they currently are.

I personally think the CCR is a really powerful model and *well worth* the investment. We have commercial licences for it at my place of work and maybe one day I'll be able to talk about that.

On the other hand, it could be that by helping solve issues both of concurrent systems design and of scalability, everyone is using it, but keeping quiet, hoping that no-one else will. But I doubt it...

CCR is not as popular as it could be due to how it requires a shift in "concpetual programming". There are many teams inside of Microsoft that have implemented some concurrent system using CCR (generally in incubation stage or experimentation). Anybody
who does anything with Microsoft Robotics is using the CCR (albeit implicitly). I think I understand your general question: "Why isn't the CCR more poplular than it appears to be?"

I will make sure to ask the creator of the CCR this question at our next interview. I know that George will probably disagree with the notion of a lack of CCR adoption, but he'll have more concrete evidence than I can come up with off the top of my head.

Very cool. I remember a thread me and evildictator had on this very idea that was similar. Grab patterns from Erlang and CCR and wrap it in a language/combiler pair that will enforce correct-by-construction design. I also like that the patterns itself
is a simpler version of STM. Essencially the R/W agents are a localized mini-STM without needing to implement a full STM. And that you can even replace it with STM under the covers as an implementation detail and the syntax does not change much and the model
remains the same. And think you still have the flex of using PTL libraries "inside" your agent. It smells like this is the right approach and abstraction going forward IMHO. Nice work, hope to use some ctp bits in future.

Also thinking... Suppose the windows UI thread loop was *replaced with a single Agent. Now we don't have to worry about IsInvokeRequired() or BeginInvoke() or even hanging the UI thread inside callback method. We have a clear seperation and threads don't
matter in terms of UI correctness. This would also mean we can forget about what thread we got a callback on (i.e. timer event) as it will not matter. The UI agent can then be free to have any type of parellelize it can correctly handle inside itself and
all is hidden from programmer. It would mean we would have to think about explicitly blocking/pausing other UI actions if you don't want people clicking same button twice before last action completes for example. A light begins to appear...

I think it's a good idea to explore how an agent-isolated model can be used to simplify expressing "ownership" of the UI. It certainly seems worth exploring in some depth.

Another interesting angle is to look at how Maestro contracts can be used to express correct ordering of UI events, to control enabling/disabling of UI elements and force some formalism on UI interactions. Don't know whether that's actually going to prove useful,
but it seems interesting to explore.