File size

File size

File size

The Concurrency and Coordination Runtime (CCR) is a lightweight port-based concurrency library for C# 2.0 developed by George Chrysanthakopoulos in the
Advanced Strategies group at Microsoft. Here, we have a deep discussion about CCR with George, a Software Architect, and Satnam Singh, Architect. You can get more info about CCR on the
CCR Wiki. This is super cool stuff and represents a really innovative approach to making managed threaded programming more readily understandable and predictable.

CCR? I was actually thinking Condition Code Registers... but I guess that's because the band was before my time. As with all other video's released lately, I've downloaded it to my file server for viewing after my last exam. Something to look forward too.

Thanks for putting out the Going Deep series Charles, I'm looking forward to it.

extremely interesting stuff! and the video was really fun to watch, too! reading through the paper i'm wondering how the port based ccr solution compares to the active/future objects idiom Herb Sutter is thinking about for c++0x/c++cli. did the ccr team
and the guys around active<> objectives share some thoughts on the topic or are these two completely independent efforts?

again a worthy installment of going deep! nice work, Charles and Scoble!

Hi, we developed the CCR independently from the futures work. We wanted to investigate a model where the asynchronous interaction is visible, not implicit. Futures have been around for awhile and we believe that we can express the patterns futures enable
easily using ports and our basic arbiters. We are not certain having ports exposed directly is the way to go, but its a concrete, useful alternative to locks and custom one-off queuing implementations we all end up writing when we want to hook up things together.

We have discussed with Herb to see if the CCR can be the execution target for the futures/active work, assuming a native version is written. The CCR right now is only available in managed environment.

Can't wait to watch this. Multicore is a very dear issue to me. It took long enough for the hardware industry to see that parrellism is the way of the future. I wonder how long until everything is distrubted computing. Imagine tapping the power of all
these machine out on the planet. As I'm typing this my dual core 2gb system is barely using any of its resources, the day when our machines are always working 24/7... and quick too, that will be a nice day.

Hope Microsoft is still around, and the leader in the industry, when this paradigm shift happens.

EDIT: Just finished watching this video, this thing, combined with the Singulairty OS makes me mouth water at what OS will be out in the next decade.

Hi,I think this library could be very helpfull, however I can't see the SDK download is it out yet? Also could you elaborate into more details (perhaps with some code or UML) on the futures/ActiveObject pattern?Thanks and keep up the great work!

Wow! Really Really Smart guys. They were/are light years ahead of me. It's like their brains are working at mach 5 speed and their mouths are only going 60 miles an hour. Most of this content went over my head but its super sexy stuff to hear about.

And I loved how robert stayed quiet and did an awesome job while filming.....but the best part was the number of time I heard Charles say "Fantanstic". I stopped counting at 10. Looks like I've stumbled onto the new buzz word around concurrency...."It's
Fantastic" should be the tag line!

Keep up the great work and showing us 9ers these really smart people and their sexy forward looking technologies

As soon as there is an SDK available, we will let you know. In the meantime, read the paper by George and Satnam, watch the video a few times, get yourself ready. There's a fair amount to get under your belt in terms of understanding what this is and how/when
to use it. C

PS: Scoble generally keeps quiet and does great camera work for Going Deeps. But, he'll always get a question or two in, which is good! I do think his antics are best served when he's interviewing product team PMs, etc

Very interesting video. Having also read the
OOPSLA/SCOOL paper on the CCR would I be right is assuming that CCR is a step above Comega (not knowing anything about Comega)? Is there any timelines for these work making it into C#?

Given the amount of research being done on the CLR, one can probably assume that somewhere around 2010, Windows will become a managed OS built using the work Microsoft Research is doing today - CCR, Comega, Singularity etc.

jskeet, you are right CSPs, Pi calculus and alot of other areas are all related. The CCR definately is influenced by previous work. Alot of the influence came from my experience in the Xbox team writing asynchronous APIs to keep things responsive and also
from my time in the NT Kernel (Base) team working on device drivers. Soon it became apparent that a nice way to write message based, asynchronous code, but without the complication of continuations, callbacks etc and with a nice way to coordinate among many
things at once (like scatter/gather) would be very useful. I also wanted to be available in C# or VB or any CLR language! Yes the CCR is accesible from VB.Net.

Check out the Wiki linked above for examples on how we used Iterators to make code look like "blocking" calls even if its full asynchronous.

Iterators in CCR are used in a very interesting way that allows you to even do loops!

If you're interested in the CCR then you might also like to look into work on software transactional memories (STMs) which are also a very promising approach for control oriented concurrency. We've written a draft paper about how to use a transactional
style of programming for concurrency and you can get the paper from
http://research.microsoft.com/~simonpj/

It's not just C# and VB.NET. The CCR is available from any language which properly handles delegates, generics, and interfaces. Most languages which run on the CLR and support the CLS and generics will be able to access the CCR.

Hi, we just think that a message passing coordination library is not the only solution out there. I think its plenty good and usable now but people might find it too "low level" or explicit. I think the CCR is complementary to other solutions like Software
Transactional Memory (STMs) if you use STMs to get isolation and protect shared memory.One good thing about the CCR is that lines up well with distributed solutions that use message passing.

While CCR has things like interleave which can easily protect access to shared data structures, a version based scheme, in some cases, will give you better results.Example:

// we have two methods that access some shared state. Unlike locks// i dont need to know what that state is in the Ccr. This way // you dont deadlock by grabing locks in the wrong order, etc

in the snippet above CCR will run as many instances of ReaderHandler as messages posted in the MainPort,as long as no instance of MyUpdate Handler is running. It will stop scheduling concurrent handlers if a exclusive handler gets queued. This is like a writer
biased reader/writer "lock". In general we use things like interleave to get incredible concurrency (millions of work items).

Our paper starts with an introduction to join patterns in the Comega langauge and we then go on to relate that to our system. You can get the paper off my website:
http://research.microsoft.com/~satnams

The joins in Comega are declarations which makes them rather static. The joins in CCR are statements and they can be dynamically constructed which makes them more flexible.

Hi Multicore, the paper linked in the header of this forum mentions C-Omega and its roots in the Join Calculus. DavidRichter also mentioned it earlier. I became aware of Join Calculus after CCR was already fully developed (I was aware of the Pi calculus
and CCS/CSP). For the CCR, the join primitive is just one of the possible coordination primitives. Our dispatching architecture and arbiters allow you to do things Join describes but our primary motivation was patterns we run into coding various applications.
We then codified those patterns. One of them happened to be join plus the variable and dynamic versions of join not described in earlier work (althoutgh they are expressible in the framework of the join calculus)

One thing I need to mention is that Join, choice etc are not rooted in the Join or Pi calculus. These are primitives OS developers and application developers have used for a very long time. In win32 they have names such WaitForMultipleObjects and the semantics
are very similar, but on event sychronization objects not queues, like we do.

All these deep CS stuff is not very approachable to most people and from the start my motivation was to improve, using a common, small set of primitives, alot of the existing OS synchronization APIs, not encode something found in theory.

I am glad you asked about debugging. We have worked on a debugging story that is message and asynchrony aware.

Unlike stack based debugging which is little to no help here (since the thread you wake up is arbitrary and the stack frames above you are just the scheduler) you need something thet logically relates your messages and the handlers which wake up to service
them. This creates an execution tree you can follow or set breakpoints on messages. Yes breakpoints on messages so no matter who picks them, you break there. In the CCR, when you post to a port, you dont know who is listening (part of the cool loose coupling
we give you) but that is a nightmare for regular debugging, since you cant "step-into". Well, with our message based debugging, you can "step-into" the receiver of the message and then any others receivers that fire because of messages logically related.

Its sounds a bit fuzzy right now, but when the CCR bits become available for download as a preview (first half of next year or sooner) a debugging story will go with it

First off, this stuff looks really cool. I look forward to trying it out.

I am a little surprised, though, that method names (e.g., Port.post) do not follow .NET Class Library Guidelines; do Krzysztof Cwalina and Brad Abrams know? Seriously, though, I hope this is corrected before the release of the SDK. The
benefit that developers receive from these common standards is very real.

Does the CCR automatically take care of distributing the work across multiple cores/processors? Or do you still have to do that yourself but the CCR just helps you?

And if the CCR does take care of it, are there ways to specify how you want that work distributed? Meaning do you have the ability to say I want thread A and B to run on processor 1 and thread C, D, E to run on processor 2?

Absolutely. CCR APIs will pass through FxCop with zero issues. So far we have been experimenting with syntax that is concise and readable but before we release, even as a CTP, it will conform to the standard MS rules.

Btw, if people read the wiki, although by far not an exhaustive reference it allows you to give feedback on the syntax we have chosen. We welcome all feedback. Underneath the CCR has a concistent design where all arbiters, receivers are schedulable tasks.

If you dont like the operators, you can always create directly the arbiter and schedule it. Example (with operators first):

// create a port with two queues, one for each message typePort<SuccessMessage,FailureMessage> Response = new Port<SuccessMessage,FailureMessage>();

// the last line above is the same as this:_pDispatcher.post( new Choice( Response.with(Success), Response.with(Failure) ));

The activate() api is of a base class that has a default dispatcher set. Programs can choose to create different dispatcher to interop well with STA threaded legacy apis or for more explicit load balancing. But they key is that the simple, consistent treatment
of everything as a scheludable task, allows you construct things the way you want, in an
imperative manner, and then issue them as commands.

Short answer: Yap we take care of it. And yes, if you want to, you can specify in more detail how the scheduling will occur.

Long answer:The CCR takes care of automatically distributing work across processors. All you do is associate receivers with ports, constrainted under some arbiter (choice, one-time/re-issue receive, interleave etc) and we will generate work items when messages arrive,
and schedule them across task queues with some number of OS threads in the back end picking them apart. Usually we use 2 threads per processor but you can set that number.

We can achieve near perfect load balancing across heterogenous tasks and keep all your CPUs hapilly busy. A CCR program just needs one instance of a dispatcher process, and if you set the number of threads to use to 0, if you want us to choose how many threads
per processor to utilize in the back end.

Dispatcher creation:

DispatcherPort MyDispatcher = Dispatcher.Create(0,"FriendlyName");

// now we can use it to associate a piece of code// that will execute in one of the CPUs, when message arrives

MyDispatcher.post(!MyMainPort.with(delegate(DoWork W){ // do work here. If you get N DoWork messages, we will // run as many versions of this code concurrently as your CPUs!}));

void Init() { // because we associated a dispatcher // when we were constructed, we can just // use the base member activate()

activate(!MyMainPort.with(delegate(DoWork W) { // do work here. If you get N DoWork messages, // we will run as many versions of this code // concurrently as your CPUs!

}));

}}

If you want greater control you can create multiple dispatchers, and then each item you want to activate a receiver, use a particular receivers. This is very powerfull and unlike global threadpool schemes: By being able to associate a dispatcher per receiver,
you can interoperate very well with Single-Threaded-Apartment components, like COM, but your CCR code does not look any different!

We will give you at least one simple "adaptor", used for UI, where you post messages on ports, but using its own dispatcher, with just one thread, it does Invoke in the back, on your behalf, toa winform.

All this helps you go the next level, if you wish, in controlling execution, scheduling etc and interoperating with the OS.

Developing dynamic analyses for message passing programs is of course important (e.g. run-time debugging). However, I feel that for message passing programs that is not going to be enough. We will also need the help of more powerful static analyses to help
catch problems like deadlocks. There is some interesting work involving
contracts and behavioral types which looks like a promising approach. For exampe:
http://www.research.microsoft.com/behave/index.htm

Another thing to note about the CCR is that it is just a library for .NET so its functionality is available to other .NET langauges. We've even written CCR programs in radicially different langauges like F# (http://research.microsoft.com/projects/ilx/fsharp.aspx)
which a functional language in the style of ML. Personally, in the long run I think we need to have langauge level support for concurrency (e.g. what we have seen with rendezvous in Ada and recent work on Sigularity
http://research.microsoft.com/os/singularity/). Then we can perform static analyses to better optimize our programs and to check them for bugs. However, in the short term the CCR provides a pragmatic
way to support certain kinds on concurrency programming in existing .NET languages.

Like everyone else, I am itching to get my hands on the library and try out some stuff. It looks like it will solve some very specific problems for me.

On a related topic, this prompted me to check out the latest C# threading books to see if there's anything good out there. Scoping Amazon and B&N only provided the "C# Threading Handbook" by Tobin Titus, et al. Might have the content, but doesn't get a good
rating (plus it is about $80 on Amazon for a 300 page book). Anyone got a better suggestion for sources on doing it "old style"?

Can you elaborate more on what you're looking for? Threading is a pretty broad topic. What problem are you trying to solve right now? For example, are you developing a massively parellelized, loosely-coupled, distributed system?

How might this help the write better multi-user server programs?I mean today you have the thread-per-socket method, which allows easier programming and correctness verify, but does not scale much beyond 1700+- threads. The async method scales better, but is a nightmare to program and reason about correctness. Even using
a thread pool, you have same kind of issue with managing state as the pure async play. Would this offer some kind of new pattern or middle ground where it would feel like a thread per user model, but actually be async in terms of threads? Any simple server
example to show idea? Thanks guys, this is great stuff!

This indeed offers a middle ground. Using iterators and anonymous delegates you dont have to explicitly pass state around or do you have to explicitly code continuations!. You can scale to hundreds of thousands of work items and the same time keep your
code in one place. Using the yield return keyword, you get "blocking" behavior but without blocking. This way you keep all the logic in one place, without jumping around continuations.

A server example is actually given in the wiki and in the examples above but i will show one here. Its so concise, its easy to miss:

// lets say you need to two things for every request that // comes in: Read from a file and send a log update. // again we have a simple stream wrapper around a file FileReadRequest fr = new FileReadRequest(); LogUpdate log = new LogUpdate();

I have an immediate need for more threading info and a future need for this concurrancy and coordination runtime.

In the near term, I'd like to be better at threading with common related ASP.NET patterns or functions. Such as: file access and update coordination, access and update of memory-cached single database objects, scheduled process coordination, and so on. I'm
expecting to discover more uses with better skills in this area, especially for the AJAX type processing or other things I haven't encountered yet as a deliverable.

In the future, I'd like to revisit some past ideas on physical system simulation that permit calculations without the level of explicit synchronization that has been required in the past. The CCR seems to be a great fit for that. If this can lead to easier
deployment to calculation on arbitrary processor/network configuration, that would be great. In fact, there are some natural network problems that would really benefit (think simulation of electrical distribution networks. Or flooding analysis on major rivers.).
So simulate a network on a network (a statement only a geek would appreciate!).

Any pointers on either topic would be appreciated. You can email me direct at Walt AT NOSPAM Lounsbery.com. Thanks!

Please note the section in John Vert's article where he mentions all the benefits of using IoCompletion ports to simplify handling I/O requests and getting just a handfull of threads to do all the work. The CCR, using one consistent interaction mechanism,
the CCR Port, provides a very similar model. You use ports to indicate completion of tasks, and you attach Reissue receivers (using activate) to deal with them. You dont care how many CPUs your system has or how many threads you need. We will do the right
thing. The key, in both schemes (NT and CCR) is asynchrony.

hmm. Nice. Thank you sir. If you ever get time, maybe you could point/post a simple message stream loop server to handle multiple users (i.e. sockets) with open connections and sending/receiving messages in a receive loop for each client. I am missing it,
but would seem to still be bound by 1700+- user connections as each active connection would take at least one main thread. Or am I wrong? Thanks much!!!--wjs

Staceyw, you will actually be able to service alot more than the number of OS threads in 32 bit systems, if you use asynchronous socket APIs, even from C#. The underlying OS will use overlapped I/O and will not block a thread per connection. Then, the total
number of simultaneous requests you can service will not be bound by OS threads. The key ofcourse is to use asynchronous APIs all the way through, so you dont consume a CCR or C# threadpool worker. YOur actuall throughput might not increase btw, but your system
will be able to service hundreds of thousands of packets, independent of each other so appear more responsive.

See one of the examples of iterators in this thread or the wiki.We actually have implemented a CCR based transport that does this (but cant post the code, yet , and it uses the C# asynchronous socket calls underneath. We just had to wrap them with CCR ports
and simple processes so we did not have to deal with Begin/End.

The CCR, using iterators allows you to get "blocking" behavior so your code does not look like the spaghetti of continuations we find ourselves writing again and again. You can actually encode a while loop , processing packets but with no blocking! One of the
examples in the wiki shows a for loop using iterators to encode asyncronous but logically sequentially req/rsp operations. It keeps it all in one routine.

So really what we are doing for you, is allow you to use the asynchronous OS APIs, through a much more readable front end, to get the logical behavior of threads, but without the overhead and limitations.

When threads become super lightweight one day( if ever), then the CCR can use alot more of them and get rid of iterators, but its syntax will not really change. The CCR can have blocking arbiters (join, choice etc)

I find CCR extremely interesting and can't wait to get my hands on it.

Is there anything similar out there that we might also want to take a look at, that is downloadable now?

The scenario I have right now is, I am building a product that uses a lot of multithreading. It is a smart client application that is using the Smart Client Offline Application Block along with a number of other threads handling various tasks like incremental
synchronization of data from the offline block, user interface updating, etc. I also have a number of singleton classes that act as gateways into different core functionality. I'm using the lock statement in C# in my singleton classes to control access to
public property setters from multiple threads. I'm looking for ways to improve performance in some areas.

One issue I'm working on right now is, there are various bits of user interface information that get updated from multiple threads. I'm using Control.BeginInvoke etc. to manage the user interface updates when some of this data changes (for instance, wireless
signal strength, online/offline status, data synchronization status information, etc.). I'm working on solving some cases when it seems there is contention to update the UI on the UI thread and it causes "pauses" that are noticeable when mousing over toolbar
icons that hot track for instance (I have a XAML toolbar where the icons "zoom" 50% when moused over).

yes UI responsiveness it can definatelly help you with, especially coordination among multiple things , like IO, while keeping the UI going.

The Win32 and Winforms UI model is a bit hard to get right however because of its built in SingleThreadApartment design and various re-entrancy issues. Right now i would recommend you look at the latest PDC talks on concurrency, especially the talk from Jan
Gray and Joe Duffy on CLR concurrency best practices. Also look into the background worker CLR 2.0 feature that helps with UI concurrency.

One issue I'm working on right now is, there are various bits of user interface information that get updated from multiple threads. I'm using Control.BeginInvoke etc. to manage the user interface updates when some of this data changes (for instance,
wireless signal strength, online/offline status, data synchronization status information, etc.). I'm working on solving some cases when it seems there is contention to update the UI on the UI thread and it causes "pauses" that are noticeable when mousing
over toolbar icons that hot track for instance (I have a XAML toolbar where the icons "zoom" 50% when moused over).

I too am looking forward. I might approach it this way:Instead of updating UI all over the place, I would have a focus point.I clean pattern I have been using is a Blocking queue. So each thread that needs to update the UI will jusend a message to the queue. Your UI thread can't be the consumer as you UI will block, so have one UI helper thread that is the single consumer of the
queue. It just pops the queue and calls BeginInvoke to update UI controls depending on the Message type. So you can have any number of producer threads all pushing the single queue - all they need is a ref to the queue. It is also a single location to signal
all producers to shutdown (i.e. stop the queue). You also have a single place to "throttle" the speed of UI updates with Thread.Sleep(). You can check my BlockingQueue<T> and samples in the Sandbox. Cheers!--William

There are hacker forces trying to break into your machine right at this very moment in order to temporarily lease your CPU and other resources on the black market. It sounds like science fiction but it isn't -- the black market is merely ahead of legitimate
business models, something akin to big pharma selling pot, which they already do.

Eventually this will be the way we "buy" computers. We'll rent computer time on machines that we never see. I think that was the idea behind the Chi Corporation in the late 60s...

Hey Charles. Think you could talk to Andrew Birrel (of MSR) who wrote "An Introdution to Programming with C# Threads" about concurrency sometime? Now that I think about it, I wonder how similar in concept CCR is to the SOA RW lock Jeffrey Richter TIA

Someone at work made me aware of that lock as well, 3 or 4 months ago. The reader/writer lock jeff wrote is a subcase of the interleave primitive we have. We have talked to jeff about it and Jeff was thinking down the same path that led to the CCR. The
CCR however normalizes all operations to be coordination of messages over ports, so reader/writer is just one of the coordination primitives you can implement cleanly. Many other things, like its dispatching model, port behavior etc are very different. Its
more of a complete runtime than a single primitive. That can be both good and bad.

Anyway, stay tuned, Jeff is working with us, giving good feedback, and in the future help with articles on the CCR.

I have to admit this looks amazing and is exactly what we need on my current project. I currently have a distributed server system that accesses data locally and on other machines, but I'm having difficulty adding parallelism to the system. I believe this
would allow me to easily add this if I can massage the dispatcher to not only load balance across processors but across machines.

Georgio do you see this as being already built in or at least a workable implementation on my part? I already have a running system (in .net 2.0) capable of handling thousands of requests a second, and I believe I can easily shoehorn this into it. We are
still months away from going live so hopefully the timing will be workable on my part to use this.

We actualy have built distributed system infrastructure on top of the CCR. The CCR itself is not ofcourse about distribution (you need other machinery for that, which sounds you have built already) but it helps you with coordinating I/O, so it was a good
fit for our distributed work. With some simple adaptors over .NET system apis (like web requests, sockets, streams) we were able to have one consistent way to deal with i/o and load balance well.

So i see no reason why you cant use this in your system.

I really want to get this out there, and have people play with it. As i said, in a couple of months you can play with the CCR, plus a bunch more that has been built on top of it Its currently in good shape and as i mentioned earlier Jeff Richter is working
with us to simplify the object model and then write up an article on it.

Yes, ports, dispatchers etc are regular C# objects so they can be used anywhere you use and store object references.

Yes ports can be extended. You just implement the appropriate interfaces or easier, derive from Port<T>. We used to have LIFO, random insert implementations but again and again FIFO became the norm due to to starving issues related to LIFO and the distruptive
attributes non-FIFO methods have on streaming or preserving sequence of events.

messages stored on ports, are by reference. We have extended some of our Arbiters (for example we have a variant of Receiver<T>) that deep clone objects before waking up the user task+ delegate associated with the port. The CCR allows you to do this type of
extensies trivially and in a manner that the sender (the process that Posts) never has to know about. As i mentioned earlier we have a distributed system built on top of it

Great! Those features really do simplify the implementation of our app. What I meant in the the first question was if the Ports are threadsafe: for instance, can I post to them from different CCR Tasks, eventually running simultaneously, without having to worry about race conditions in the Port's object, right?

Hi, the CCR is ready for its first public release. It will be available tuesday as part of a new product CTP we will announce then. It is an independent part, so people just interested in the CCR can use it by itself. I will post the link tuesday. Also
expect another channel9 video showing the ccr in action

ok the bits are live. You can go to msdn.microsoft.com/robotics, and download the setup. its kind of big since it includes a few external components but once it installs, the CCR dll will be under \bin. Take a look at the service tutorials that come as
part of the robotics SDK. The CCR is used as part of a service model that will give you cross machine distribution, in addition to concurrency and coordination within a machine.

July 9th you will also get a MSDN article ont he CCR with sample code using it in various scenarios.

Very enjoyable video! I have a simplistic question -- I thought a "runtime" was something that loaded code and managed its execution (e.g. CLR, Java runtime). This is a type library -- what makes it a "runtime"? Thanks!

﻿Very enjoyable video! I have a simplistic question -- I thought a "runtime" was something that loaded code and managed its execution (e.g. CLR, Java runtime). This is a type library -- what makes it a "runtime"? Thanks!

The CCR, even if its a library, actually manages all execution, replacing the CLR thread pool or any manual threading code. Underneath ofcourse it uses the OS+CLR.

The CCR manages messages on ports, task load balancing and execution, iterator task stepping, dispatcher lifetime etc. So its somewhere between a library and an execution environment. If you check out our robotics video, it might make it a little bit more clear
why the CCR is a fundemental piece of our execution environment

Great stuff! Very inspiring. In the video a small demo is shown of the 'concurrent msbuild' application the team build with the ccr. I would like to know if it's possible to recieve a copy of this application or if it might be released in some shape or
form in the near future.

Remove this comment

Remove this thread

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation,
please create a new thread in our Forums, or
Contact Us and let us know.