Andrew Arnotthttps://blogs.msdn.microsoft.com/andrewarnottms
News from my corner of the Visual Studio Connected Experience team, programming tips, and solutions to common programming issues.Thu, 11 May 2017 16:12:14 +0000en-UShourly1Limiting concurrency for faster and more responsive appshttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/uwTh2MTQk1s/
https://blogs.msdn.microsoft.com/andrewarnottms/2017/05/11/limiting-concurrency-for-faster-and-more-responsive-apps/#commentsThu, 11 May 2017 16:12:14 +0000https://blogs.msdn.microsoft.com/andrewarnottms/?p=1115Read more]]>When you have a set of highly parallelizeable work, executing it concurrently can be easy:

Of course you’d probably want to track the work at least so you know when it’s done:

Calling Task.Run schedules the work to run on the .NET ThreadPool which is highly tuned and can likely get the work done as fast as you have CPUs to do the work as it tends to schedule at least as many threads as the machine has CPU cores.
The problem with the above code is that it floods the threadpool queue with items all at once. So if you have 10 items on your list, and only two cores, the queue and threads look like this:

Queue Active
XXXXXXXX XX

Now suppose after you add all this work to the queue, your app adds one more item Y to the queue:

Queue Active
YXXXXXXXX XX

Notice its placement is *behind* all that work you previously enqueued.
When Y is more urgent than X, you just have to wait till all the X items are done anyway before Y gets time:

Queue Active
YXXXX XX
Queue Active
YX XX
Queue Active
YX

That can be very bad for application responsiveness or perceived performance if the user is waiting for Y but doesn’t care (as much) how long X takes.

We can solve this by applying a simple throttle to the generator of X tasks, such that X items are progressively enqueued over time so that Y can get attention more quickly. If we assume that X has already started, the timeline on the queue may look like this:

Do you see how the queue is never empty (guaranteeing all cores are still busy) but Y got through much more quickly? It’s a fairness scheme. And it ensures that X still runs very quickly in the absence of other work, and when other work exists, it shares the threadpool much more equitably.

Encoding this is actually quite simple if we use the ConcurrentExclusiveSchedulerPair class. Suppose our goal is to keep all cores busy while maintaining a short threadpool queue, no matter how many items of work you have to complete. Just a few extra lines is all we need to achieve our fairness scheme:

In the above, we’re using a concurrent TaskScheduler that comes with .NET that allows us to throttle the rate at which work is passed to the ThreadPool. By scheduling items for twice as many cores as the machine has, we will maximize CPU usage and have one item on the queue for each, ready and waiting to go as soon as each of the threadpool threads finishes work. As each work item completes, the TaskScheduler we’re using will add another item to the threadpool’s queue, ensuring that it stays busy. But importantly, it also keeps the threadpool queue short at any given time, ensuring that our Y task that comes in from another source can be executed in a reasonably short time.

The ConcurrentExclusiveSchedulerPair approach works great if you’re executing synchronous code, or async code that does not use .ConfigureAwait(false) everywhere. Otherwise, throttling async code that uses .ConfigureAwait(false) requires another technique like use of SemaphoreSlim.

To drive the point home that throttling your concurrency with this technique achieves better responsiveness without sacrificing throughput of your work, I wrote a little console app that schedules the same work both with and without throttling. Then I wrote it again to demonstrate an async work variant.

I find it very interesting the throttled run is even faster in these tests. Your mileage may vary, but what is very clear and will be consistent is that you can see the threadpool is much more responsive to other work in the throttled run.

And that my friends, is the point I’d like you to take from this post.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2017/05/11/limiting-concurrency-for-faster-and-more-responsive-apps/feed/2https://blogs.msdn.microsoft.com/andrewarnottms/2017/05/11/limiting-concurrency-for-faster-and-more-responsive-apps/TaskSchedulers and semaphoreshttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/p1HSV_0VdmM/
https://blogs.msdn.microsoft.com/andrewarnottms/2016/02/06/taskschedulers-and-semaphores/#respondSat, 06 Feb 2016 21:40:39 +0000https://blogs.msdn.microsoft.com/andrewarnottms/?p=1103Read more]]>When you write multi-threaded code, it’s important to be aware of whether the code in other libraries you call into is also thread-safe. By my observation, most code written is not thread-safe. So if you’re writing thread-safe code, kudos to you. But as you’ll sometimes need to call non-thread-safe code from your multi-threaded code, this post presents a couple options for doing just that.

Writing thread-safe code is not easy. Even if you think you’re writing thread-safe code you may even have bugs in it. In fact even in the .NET Framework, most types are not thread-safe on their instance members, but are thread-safe for static members. These details are in MSDN documentation for each type and member. Any library’s documentation should discuss how thread-safe it is (or isn’t). If the docs don’t discuss it, you should assume it is not thread-safe.

A less common solution might be to create a dedicated thread for accessing non-thread-safe code. This may be required when the non-thread-safe code is a native COM object that actually requires the exact same thread be used for every call. But otherwise it is unnecessary overhead to create such a thread, since the constraint isn’t that you use exactly one thread, but merely one thread at-a-time.

A very common way to call into code that is not thread-safe is similar to how you may be doing your own thread-safe code: wrap it with an exclusive lock. Locks can be very lightweight and are often the right choice when you don’t expect lock contention to occur frequently or last a long time.

If the code you are writing is or can be async, you have at least a couple other options.

Semaphores

In C#, the lock statement is very similar to a semaphore created with a count of 1. Your multi-threaded code can enter such a semaphore before accessing non-thread-safe code as a way to allow your multi-threaded code to safely call it.

In .NET, the SemaphoreSlim class offers a WaitAsync method you can await on to efficiently yield your thread if the semaphore is not immediately available. Consider this example:

Locks will synchronously block until they’re available, but a semaphore can be awaited on. Also, semaphores can be held across asynchronous awaits, which C# lock won’t allow you to do. Another rather unique feature of using semaphores is you can hold them without holding a thread. So you can, for example await on something while holding a semaphore. It may not always be a good idea to hold a semaphore for a long period of time, but it can be powerful when you need it as it guarantees that no one else can enter the semaphore and observe your object’s state while it’s in the middle of an async operation. So for example, we can do this:

Exclusive TaskSchedulers

Another technique for ensuring that code executes only on one thread at a time is to use a TaskScheduler with a policy to only execute one Task at once. .NET’s TaskScheduler.Default will schedule work by adding it to the threadpool queue, which means it may run multiple tasks concurrently. But .NET also offers a TaskScheduler that ensures that only one Task executes at a time: ConcurrentExclusiveSchedulerPair. Consider the thread-safe dictionary example used above, but written in terms of an exclusive TaskScheduler:

Here, our thread-safe AddAsync method schedules a Task on an exclusive TaskScheduler. As soon as that TaskScheduler is not busy, it will schedule our task for execution.

.NET’s ConcurrentExclusiveSchedulerPair class can actually serve a broader purpose. If your code can allow concurrent access for reading data, but requires exclusive access for writing data, the scheduler pair can help you achieve that. You schedule writes to the exclusive scheduler, and reads to the concurrent scheduler. This scheduler pair work together to ensure that Tasks sent to each scheduler only execute when they should.

But the above example isn’t particularly pretty. And the Task we schedule is synchronous. How can we make it asynchronous? The fastest way to do that based on what we already have is simply change the delegate we pass in to an async delegate and (very important!) add .Unwrap() at the end:

The Unwrap() is important because TaskFactory.StartNew doesn’t recognize async delegates, so StartNew will return a Task where the outer one completes as soon as the async delegate yields, while the inner Task completes after the async delegate fully executes. So what you must return to your caller is the inner Task. Unwrap() does this for you.

There is one significant difference in functionality between what we now have and the last SemaphoreSlim example. Both examples yield between the Add and Remove calls. But in the semaphore example no one else can execute within the semaphore during the yielding Delay(), whereas in the TaskScheduler example, other code *can* execute during the Delay(). Why the difference? TaskSchedulers do not have any insight into async work. They only execute synchronous delegates — not async ones. Each time your async delegate yields (such as for the await Delay()), the delegate has returned back to the TaskScheduler, which considers that Task completed. After the delay has elapsed, another Task is scheduled (on the same TaskScheduler) to execute the next (and last) synchronous portion of your async delegate.

Async methods tend to stay in the context they were started in. Or in other words, when you await a Task, the context you are currently running in is captured so that when the Task completes, it can resume your async method execution in the same context it started in. What is a ‘context’? It entails a lot, but for our purposes let us say that .NET first checks for SynchronizationContext.Current and schedules continuations using SynchronizationContext.Post if it finds one, and otherwise uses TaskScheduler.Current, which in our case is the exclusive TaskScheduler but otherwise would be the same as TaskScheduler.Default, which allows concurrent execution on the threadpool.

Should you use SemaphoreSlim or TaskScheduler for your synchronization needs? As just presented, the most important factor may be whether you need to block others from executing code across asynchronous code. But if that isn’t relevant (e.g. you’re fully synchronous in these critical sections of code), you can go with whichever one lets you write code more naturally.

It’s worth calling out that neither approach guarantees that your thread-safe code will execute on the same thread all the time, nor that when resuming from an await that you’ll be on the same thread you were before the await. They merely guarantee that your code won’t execute with any other code using the same synchronization object (with their varying degrees of granularity as discussed).

Making it look elegant

Our examples thus far have required that we write a lot of boilerplate code. We can do better, both for the semaphore and the TaskScheduler case. In both cases, it’s by adding the Microsoft.VisualStudio.Threading package to your project.

Install-Package Microsoft.VisualStudio.Threading

Then add this statement to the top of your source file:

using Microsoft.VisualStudio.Threading;

Now both examples are downright elegant. First, using AsyncSemaphore and its using block support:

And second, using the ability to await on a TaskScheduler:

Both samples now look quite nice. One thing we lose in the last TaskScheduler sample is observing the CancellationToken while we await on the TaskScheduler. So we make up for that as best we can by responding to cancellation as soon as we resume execution.

There are of course other ways to solve these problems. These techniques are perhaps lesser known, but often good choices for you to consider in your code.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2016/02/06/taskschedulers-and-semaphores/feed/0https://blogs.msdn.microsoft.com/andrewarnottms/2016/02/06/taskschedulers-and-semaphores/When TPL Task continuations are inlinedhttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/MmLk4046aRI/
https://blogs.msdn.microsoft.com/andrewarnottms/2015/12/31/when-tpl-task-continuations-are-inlined-and-shouldnt-be/#commentsThu, 31 Dec 2015 16:15:10 +0000https://blogs.msdn.microsoft.com/andrewarnottms/?p=1081Read more]]>Task and Task in .NET 4.0 are absolutely awesome types and they provide the basis for async support that came in .NET 4.5. One somewhat unexpected behavior they have that can occasionally cause bugs in your code is that when a Task completes, it can execute continuations inline. In this post, I explain this and what you can do to avoid it when it becomes problematic.

First let’s explain what we mean by “executing continuations inline”. Consider this code snippet:

When executed, this output is usually produced:

OTHER TASK: awaiting task on thread 3
MAIN THREAD: About to complete task on thread 1
OTHER TASK: Resuming on thread 1
MAIN THREAD: Just returned from completing Task.

Notice in particular that the “OTHER TASK” resumes execution on the main thread, preventing the main thread from executing its own next statement until OTHER TASK yields or completes. When the main thread completed the task, TPL went through several checks including available stack space on the calling thread, and decided that some continuations should be executed inline with the caller. Sometimes it makes the opposite decision, and merely schedules these continuations to execute on their own (typically on the threadpool, or on the main thread’s main message loop).

Scheduling continuations to run on their own is generally safe. So why inline continuations at all? Speed. Scheduling work to happen on the threadpool takes time, and in a multi-threaded and/or async application that time can make a perceivable difference to the user when they pile up. With the advent of the async keyword, these continuations are often very lightweight and inlining can provide a nice boost in performance. For folks who are familiar with TaskContinuationOptions.ExecuteSynchronously, it’s important to note that this flag is merely a suggestion to inline the continuation. But continuations may or may not inline whether this flag is set or not.

What about when it is not safe to inline continuations? TPL can’t predict what a continuation will execute — it only knows a few environmental indicators such as available stack space. For instance, if the thread is already near a full stack, it’s not safe to optimistically inline a bunch of new arbitrary work so inlining continuations is less likely to happen. So what if you as the programmer know that continuations shouldn’t be inlined?

You have several options to prevent inlining continuations. Some are at the task completion source, and others are on the continuation side. Here they are, in order of most preferred to least preferred:

TaskCreationOptions.RunContinuationsAsynchronously
The most preferred solution is available if you can target .NET 4.6: Create your TaskCompletionSource with the TaskCreationOptions.RunContinuationsAsynchronously flag. This guarantees these continuations won’t be inlined.

Use AsyncManualResetEvent from the Microsoft.VisualStudio.Threading library. This automatically prevents inlining continuations (including awaiters) and works on .NET 4.5 and later. And its API is slightly friendlier for representing events than TaskCompletionSource (since you don’t have to make up a return type and value that you don’t use).

Call SetResult inside another Task body:Task.Run(() => _taskCompletionSource.SetResult(0));
Inlining can still happen, but it doesn’t happen on your thread and won’t block your own method’s progress. It does mean that the Task itself may not have transitioned to its RanToCompletion state before your calling method returns, which might be a significant behavioral change for you. If that behavioral change is intolerable, you could add this line below the Task.Run line given above:_taskCompletionSource.Task.Wait();
That would ensure you block until the Task transitions and thus complete before your method returns. However I’ve seen deadlocks because of this in at least one case where there was a very interesting dependency loop and set of inlined tasks. So I don’t recommend doing this.

If you control all folks awaiting on the Task, you can await Task.Yield(); after awaiting the Task to ensure that if they are inlined, they yield it right back to prevent code to execute on the completer’s callstack.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2015/12/31/when-tpl-task-continuations-are-inlined-and-shouldnt-be/feed/2https://blogs.msdn.microsoft.com/andrewarnottms/2015/12/31/when-tpl-task-continuations-are-inlined-and-shouldnt-be/Install Android SDKs using Powershellhttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/3T7RkkGzoYQ/
https://blogs.msdn.microsoft.com/andrewarnottms/2015/10/12/install-android-sdks-using-powershell/#respondMon, 12 Oct 2015 04:21:03 +0000https://blogs.msdn.microsoft.com/andrewarnottms/?p=1072Read more]]>The Android SDK Manager requires a GUI and tedious searching for the SDKs that you want to install. There is a command line android.bat script that lets you install and uninstall these without the GUI but it’s based on an ordinal index for each SDK so you have to run the script once, note the index(es) of the SDKs you want to install, then run another command. This of course can be automated. I have done so with this Powershell script.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2015/10/12/install-android-sdks-using-powershell/feed/0https://blogs.msdn.microsoft.com/andrewarnottms/2015/10/12/install-android-sdks-using-powershell/ImmutableObjectGraph has some big updates now, and more coming soonhttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/P3r_Hg5ROjw/
https://blogs.msdn.microsoft.com/andrewarnottms/2014/11/19/immutableobjectgraph-has-some-big-updates-now-and-more-coming-soon/#respondWed, 19 Nov 2014 21:24:39 +0000https://blogs.msdn.microsoft.com/andrewarnottms/2014/11/19/immutableobjectgraph-has-some-big-updates-now-and-more-coming-soon/Read more]]>I was able to work out a model for continued open source development of ImmutableObjectGraph — at least in a limited way. This means that the 16 month silence on the public repo is over, and I was able to push a bunch of the commits I had been keeping internal to the public.

For those of you not already familiar with the library/tool, you can catch up by reading these past blog posts:

So what’s new in these 233 new commits? Mostly the generation of structs that provide a convenient means to reference an object in the middle of a recursive graph, modify it (via copy on write) and have it automatically rewrite the spine so that you have a new tree of everything, but reusing as many objects as possible. I think it’s pretty slick. Try it out and send your feedback and pull requests!

Also very exciting is some work to revise the technology this tool uses from T4 to using Roslyn. Check out a video that shows the before and after experience and again, send your feedback.This work will also be available in the open source project under the Roslyn branch shortly.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2014/11/19/immutableobjectgraph-has-some-big-updates-now-and-more-coming-soon/feed/0https://blogs.msdn.microsoft.com/andrewarnottms/2014/11/19/immutableobjectgraph-has-some-big-updates-now-and-more-coming-soon/So many exceptions… but only one can throwhttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/sS5H86P0r3Y/
https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/13/so-many-exceptions-but-only-one-can-throw/#commentsTue, 13 May 2014 07:40:33 +0000https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/13/so-many-exceptions-but-only-one-can-throw/Read more]]>When a method may throw for more than one reason, the thoughtful programmer might ask “which exception should be thrown?”

Consider a method which performs argument validation, is cancelable, and also might throw based on the state of the object. What order should these validations occur so that the best exception is thrown? Here is my own answer to that question, as an in-order list of verifications that prioritize the exception types and reasons:

ArgumentException – always validate arguments first (assuming it’s a cheap operation). Regardless of the cancellation token or the state of the object, if the caller ever passes in invalid inputs, it’s a bug in their code and you don’t want to miss an opportunity to throw to help them notice their bug.

OperationCanceledException – It’s quite acceptable that the caller knows your object is in an invalid state (disposed, corrupted, etc.) and with the transition to that state they canceled the token that came into your method. So throwing OperationCanceledException is better than throwing some other exception about the invalid state of your object because they presumably already know about that.

ObjectDisposedException – at this point, if the object is already disposed, it’s time to let them know using the best exception for it.

InvalidOperationException – if your object is in any other state that isn’t prepared for this call, throw this with a helpful exception message describing why it’s inappropriate.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/13/so-many-exceptions-but-only-one-can-throw/feed/2https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/13/so-many-exceptions-but-only-one-can-throw/Asynchronous and multithreaded programming within VS using the JoinableTaskFactoryhttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/9cLzheTqV9c/
https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/07/asynchronous-and-multithreaded-programming-within-vs-using-the-joinabletaskfactory/#commentsWed, 07 May 2014 09:07:00 +0000https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/07/asynchronous-and-multithreaded-programming-within-vs-using-the-joinabletaskfactory/Read more]]>Everyone appreciates a fast and responsive UI, and Visual Studio is no exception. Extensions that run in Visual Studio play a significant role in how responsive the IDE will be for its users. Visual Studio has been evolving over the past few cycles to not only improve performance, but also responsiveness during operations that may take a while to execute, offering cancellation or the ability to run these operations in the background while you can interact with the IDE in the meantime.

IDE responsiveness during long-running operations requires these operations to be written asynchronously or off the UI thread, which can be challenging. Although it might be easy to write and maintain async code that uses the C#/VB async keyword for responsiveness during these long running operations, doing so can cause deadlocks if that async code is ever called by a method that must synchronously block until the async work has completed. For example, code as simple as this would deadlock if run on the UI thread of any GUI app:

It can be very tempting to write code such as the above so that you can call DoSomethingAsync() most of the time to provide a responsive UI, but call DoSomething() when you have to do it synchronously. In fact completing something synchronously is quite often necessary in VS to satisfy old IVs* interfaces that were not designed with async in mind. So how do you write async code that won’t deadlock when it must synchronously block the UI thread?

In this post, we outline modern guidelines for Visual Studio 2013 for managed code developers writing VS extensions regarding the use of async and multi-threaded code that avoids pitfalls such as the one above. We’ll start with a short history lesson in COM that may help explain why the above code deadlocks. Then we prescribe the tools and coding patterns to use to avoid these pitfalls.

A small history lesson in COM thread marshaling

With Visual Studio 2010 came the introduction of significant chunks of managed code to the Visual Studio product itself. The Visual C++ project system was the first project system to be (mostly) rewritten in (ironically) managed code. This was also the version when the text editor was rewritten in managed code. Since then Solution Explorer has been rewritten in managed code and the Javascript project system was introduced as an all-managed project system.

With that managed code came subtle but important differences in how services behaved and interacted, notwithstanding backward compatibility being a firm pillar. Relevant to this post are differences in how threading rules between components were implemented.

When everything was C++ native code, COM ensured that almost everything happened on the main STA thread (i.e. the UI thread). If code running in another apartment (e.g. a background thread) called any of these COM components the background thread would block while the call was re-issued on the main thread. This protected the COM component from having to deal with concurrent execution, but left it open to reentrancy (being invoked while in an outbound call). This technique worked whether the caller was managed (automatically) or native code (via the proxy stub that COM would generate for the caller).

When those same COM components were rewritten in managed code (C# in most cases) some of these automatic thread marshaling behaviors became less certain. For instance, if native code called the rewritten managed COM component, then it would execute on the main thread since the native caller was typically already on the main thread, or since native code calls managed code through a COM proxy the call could get marshaled. But if the caller and new service are both written in managed code, the CLR removes the COM marshaling interop boundary between the two components to improve performance. This removal of the interop boundary meant that any assurance that the managed COM service might otherwise have of always executing on the UI thread was no longer guaranteed. As a result, the conscientious managed code developer writing VS components should either write thread-safe code or be sure that every public entrypoint marshals to the UI thread explicitly before invoking any internal code to help assure thread-safety.

When everything was COM written in native code, marshaling to the UI thread was done by posting a message to the windows message queue for the main thread and then blocking the calling thread until the call completes. The main thread would pick up the message in normal course of its message pump, execute the code, and then return to the message pump. In some cases the main thread was busy, and these messages would just wait until the main thread returned to its message pump. In a few cases, this work on the main thread was actually blocking the main thread from returning to its message pump waiting for some background thread to complete its work, which in turn was blocked waiting for the main thread to do some work. This deadlock would be broken by the main thread waiting using CoWaitForMultipleHandles, which ran a filtered message pump that only processed messages with a matching “COM logical thread ID” that let it know it was related work and presumably necessary to execute to avoid deadlocks.

Switching to the UI thread

When managed code needs to marshal a call to the UI thread in Visual Studio, ultimately the same approach would be taken as in native code. If you’re on a background thread, switching to the UI thread required adding a message to the message queue and (usually) waiting for it to be executed and handling the result. But at an actual coding level this tended to surface in either of two ways: SynchronizationContext.Post for asynchronous invocation, or relying on a truly native COM component to marshal the call to the UI thread and then call the managed code back from the new thread.

In fact one of the simplest ways of getting to the UI thread for a managed develop in VS has been to use ThreadHelper.Invoke. Internally this uses the method of calling a native COM service in order to get to the UI thread and then it invokes your delegate.

The problems start when the deadlock resolving code kicks in. The COM logical thread ID doesn’t automatically propagate for managed code like it does for native code. So the VS filtered message pump doesn’t know which marshaling messages to execute when the main thread is blocked in managed code in order to avoid deadlocks. So it lets them all in. Well, almost. Posted messages (in the SynchronizationContext.Post sense) don’t get in, but all the “RPC” level marshaling calls do get in regardless of their actual relevance to what the main thread is waiting on.

The one or two fundamental ways to get to the UI thread notwithstanding, there were at least a dozen ways to get to the UI thread in VS (each having slightly different behaviors, priorities, reentrancy levels, etc.) This made it very difficult for code to choose which method was appropriate, and often required that the code had complete knowledge of what scenario it was called in, which made it impossible to get right when the same code executed in multiple scenarios.

Reentrancy

Because of this nearly wide open policy for executing code from other threads, an evil we call “reentrancy” occurs when the main thread is blocked waiting for something and something unrelated from another thread jumps in and begins executing unrelated work. The main thread may have been blocked on anything (e.g. it could be a contested lock, I/O, or actually a background thread) and suddenly it’s executing something completely unrelated. When that work never calls back into your component, the problem is merely annoying and can slow down your own code because it can’t resume execution until the offending party gets off your callstack. But if that work eventually calls into the same component that it interrupted, the results can be devastating. Your component may be ‘thread safe’ in the sense that it always executes on the UI thread, but reentrancy poses another threat to your data integrity. Consider this case:

Code inspection may not suggest that this code is vulnerable to threading issues. But the call to File.Open may result in the main thread blocking. During this block an RPC call may re-enter the UI thread, and call this same method. This second execution of the method will also satisfy the file open count test and start opening a file. It will assign the result to the last element in the array, increment the field, and exit. Finally, the original call (that was interrupted) will finish its File.Open call, and then throw when assigning the result to the array since m_filesOpened is now out of bounds (beyond the last element of the array). This is remarkably similar to multi-threaded concurrency issues, but remarkably can be reproduced even though your method only ran on the UI thread, which allowed reentrancy.

Product crashes and hangs can occur due to reentrancy when code wasn’t prepared for it due to the data corruption reentrancy can cause. And these symptoms often are detected long after the reentrancy has occurred, making it very difficult when you’re analyzing the results of the devastation to figure out how it was introduced in the first place.

Lions and Tigers and Bears, oh my!

So we have two evils: deadlocks and reentrancy. Without reentrancy to the main thread we have deadlocks when the main thread is blocked on background threads that are in turn blocked on the main thread. And with reentrancy we tend to get too much reentrancy leading to corruption, crashes and hangs. As early as Dev10, VS architects would meet to discuss the current ‘balance’ between reentrancy and deadlocks to resolve some of the ship-blocking bugs that would plague that version of VS.

Avoiding this reentrancy involves turning two knobs: what kinds of messages are allowed through the message filter and/or the priority those messages themselves come in with. Letting fewer messages in tends to create deadlocks, whereas letting in more messages tends to create more reentrancy with their own crashes and hangs. Since adjusting the overall policy in the message filter was rife with dangers on either side, each cycle we tended to fix the major bugs by some localized code change that relied heavily on the other players’ current behavior and was therefore fragile – leading to yet another meeting and code change later that cycle or in the next one.

Clearly there was a need for a systemic fix. The native COM logical thread ID was attractive but wasn’t an option for managed code as far as we could see.

Asynchronous programming in managed code

Opportunities to write async code within VS tended to be few and far between, since most of the time the code implemented some IVs* interface that had synchronous method signatures such that postponing the work till later isn’t an option. In a very few cases (such as async project load) it was possible, but required use of the VS Task Library which was designed to work from native code rather than something that felt more C# friendly. Use of the managed-friendly TPL Tasks library that shipped with .NET 4.0 was possible in some cases, but often led to deadlocks because TPL Tasks lack of any dependency chain analysis that would avoid deadlocks with the main thread. While the VS Task Library could avoid the deadlocks, it demanded that the code be VS specific instead of being rehostable inside and outside Visual Studio.

In more recent history a new pattern emerged in code that ran in the Visual Studio process: use of the C#/VB async keyword. This keyword makes writing asynchronous code simple and expressive, but didn’t work at all with the VS Task Library. Since so much code has to execute synchronously on the main thread in order to implement some IVs* interface, writing asynchronous code usually meant you’d deadlock in VS when the async Task was synchronously blocked on using Task.Wait() or Task.Result.

Introducing the JoinableTaskFactory

To solve all of these problems (deadlocks, reentrancy, and async), we are pleased to introduce the JoinableTaskFactory and related classes. “Joinable tasks” are tasks that know their dependency chain, which is calculated and updated dynamically with the natural execution of code, and can mitigate deadlocks when the UI thread blocks on their completion. They block unwanted reentrancy by turning off the message pump completely, but avoid deadlocks by knowing their own dependency chain and allowing related work in by a private channel to the UI thread. Code written with the C# async keyword also (mostly) just works unmodified, when originally invoked using the JoinableTaskFactory.

So in essence, we’ve finally solved the classic deadlock vs. reentrancy problem. There is now just one way we recommend to get to the UI thread that works all the time. And you can now write natural C# async code in VS components so by leveraging async we can improve responsiveness in the IDE so developers don’t see so many “please wait” dialogs. Goodness.

Let’s look at some concrete examples. Please note that most of these examples requires that you add a reference to Microsoft.VisualStudio.Threading.dll and add the following line to your source file:

using Microsoft.VisualStudio.Threading;

Switch to and from the UI thread in an asynchronous method

Notice how the method retains thread affinity across awaits of normal async methods. You can switch to the main thread and it sticks. Then you switch to a threadpool thread and it likewise sticks. If you’re already on the kind of thread that your code asks to switch to, the code effectively no-ops and your method continues without yielding.

The implementation of async methods you call (such as DoSomethingAsync or SaveWorkToDiskAsync) does not impact the thread of the calling method. For example suppose in the sample above, SaveWorkToDiskAsync() was implemented to switch to the UI thread for some of its work. When SaveWorkToDiskAsync() completes its work and PerformDataAnalysisAsync() resumes execution, it will be on the same type of thread it was before, which is the threadpool in our case. This is very nice for information hiding. When writing async code, you can use whatever threads you want, and your caller needn’t be aware of or impacted by it.

At this point we recommend writing async code whenever you have an opportunity to. If you’re writing a method that does I/O (whether disk or network access), take advantage of .NET’s async APIs. Obviously if the main thread is doing async I/O the system is more responsive because the message pump can be running while the I/O is in progress. But perhaps less obvious is why async I/O is advantageous even if you’re already on a threadpool thread. The threadpool is a scarce resource too. By default the CLR only provides as many threadpool threads as the user has cores. This often means 4 threads, but can be as low as 1 or 2 on netbooks (and yes, some Visual Studio customers develop on netbooks). So blocking a threadpool thread for more than a very brief time can delay the threadpool from serving other requests, sometimes for very long periods. If the main thread, which is active while you’re doing I/O on a threadpool thread, processes a message that requires use of the threadpool, you may end up being responsible for the IDE freezing up on the user because you’re blocking the threadpool. If you use await whenever you can even on threadpool threads, then those threads can return to the pool during your async operation and serve these other requests, keeping the overall application responsive and your extensions acting snappy, so customers don’t uninstall your extension because it degrades the IDE.

Call async methods from synchronous methods without deadlocking

In a pure async world, you’re home free just writing “async Task” methods. But what if one of your callers is not async and cannot be changed to be async? There are valid cases for this, such as when your caller is implementing a public interface that has already shipped. If you have ever tried to call an async method from a synchronous one, you may have tried forcing synchronous execution by calling Task.Wait() or Task.Result on the Task or Task<T> returned from the async method. And you probably found that it deadlocked. You can use the JoinableTaskFactory.Run method to avoid deadlocks in these cases:

The above Run method will block the calling thread until SomeOperationAsync() has completed. You can also return a value computed from the async method to your caller with Run<T>:

Calling the Run method is equivalent to calling the RunAsync method and then calling JoinableTask.Join on its result. This way, you can potentially kick off work asynchronously and then later block the UI thread if you need to while it completes.

The 3 threading rules

Using the JoinableTaskFactory requires that you follow three rules in the managed code that you write:

If a method has certain thread apartment requirements (STA or MTA) it must either:

Have an asynchronous signature, and asynchronously marshal to the appropriate thread if it isn’t originally invoked on a compatible thread. The recommended means of switching to the main thread is:

OR

Have a synchronous signature, and throw an exception when called on the wrong thread.

In particular, no method is allowed to synchronously marshal work to another thread (blocking while that work is done). Synchronous blocks in general are to be avoided whenever possible.See the Appendix section for tips on identifying when this is necessary.

When an implementation of an already-shipped public API must call asynchronous code and block for its completion, it must do so by following this simple pattern:

If ever awaiting work that was started earlier, that work must be Joined. For example, one service kicks off some asynchronous work that may later become synchronously blocking:

Note however that this extra step is not necessary when awaiting is done immediately after kicking off an asynchronous operation.In particular, no method should call .Wait() or .Result on an incomplete task.

A failure to follow any of the above rules may result in your code causing Visual Studio to deadlock. Analyzing deadlocks with the debugger when synchronous code is involved is usually pretty straightforward because there are usually two threads involved (one being the UI thread) and it’s easy to see callstacks that tell the story of how it happened. When writing asynchronous code, analyzing deadlocks requires a new set of skills, which we may document in a follow-up post on this blog if there is interest.

JoinableTask Interop with the VS Task Library (IVsTask)

What is the VS Task Library?

The VS Task Library, if you’re not already familiar with it, was introduced in Visual Studio 2012 in order to provide multi-threaded task scheduling to native code running in Visual Studio. This is how ASL (asynchronous solution load) was built for all the project systems that were written in native code. The VS Task Library is itself based on TPL, and the COM interfaces look very similar to the TPL public surface area. In particular, the VS Task Library is based on TPL as it was in .NET 4.0, so strictly speaking the VS library doesn’t help you write asynchronous tasks using async methods. But it does let you schedule synchronous methods for execution on background threads with continuations on the main thread, thereby achieving an async-like effect, just with a bit more work.

While we’re on the subject, a word of caution: even though TPL and the VS Task Library look similar, they don’t always behave the same way. The VS Task Library changes the behavior of some things like cancellation and task completion in subtle but important ways. If you’re already familiar with TPL, don’t make assumptions about how the VS Task Library works.

One important difference between TPL and IVsTask is that if you block the main thread by calling Task.Wait() or Task.Result on an incomplete task, you’ll likely deadlock. But if you call IVsTask.Wait() or IVsTask.GetResult(), VS will intelligently schedule tasks that require the UI thread to avoid deadlocks in most cases. This deadlock-resolving trait is similar to the JoinableTask.Join() method.

JoinableTask and IVsTask working together

Suppose you need to implement a COM interface that requires your method to return an IVsTask. This is an opportunity for you to implement the method asynchronously to improve IDE responsiveness. But creating an IVsTask directly is tedious, especially when the work is actually asynchronous. Now with JoinableTaskFactory it’s easy using the RunAsyncAsVsTask method. If all you have is a JoinableTask instance, you can readily port it to an IVsTask using the JoinableTask.AsVsTask() extension method. But let’s look at some samples that use RunAsyncAsVsTask, as that is preferable since it supports cancellation.

What we’ve done here is created an IVsTask using the JoinableTaskFactory. Any work that requires the UI thread will be scheduled using background priority (which means any user input in the message queue is processed before this code starts executing on the UI thread). Several priorities are available and are described on MSDN. The cancellation token passed to the delegate is wired up to the IVsTask.Cancel() method so that if anyone calls it, you can wrap up your work and exit early.

Here is a longer, commented sample so you can understand more of the execution flow:

Which scheduling library should I use?

When you’re writing native code, the VS Task Library is your only option. If you’re writing managed code in VS, you now have three options, each with some benefits:

System.Threading.Tasks.Task

Pros:

Built into .NET 4.x so code you write can run in any managed process (not just VS).

C# 5 has built-in support for creating and awaiting them in async methods.

Cons

Quickly deadlocks when the UI thread blocks on a Task’s completion.

Microsoft.VisualStudio.Shell.Interop.IVsTask

Pros:

Automatically resolves deadlocks.

Compatible with C++ and COM interfaces and is therefore used for any public IVs* interface that must be accessible to native code.

Cons:

Relatively high scheduling overhead. Usually not significant though if your scheduled work is substantial per IVsTask.

Hardest option for managed coders since tasks and continuations have to be manually stitched together.

No support for async delegates.

Microsoft.VisualStudio.Threading.JoinableTask

Pros:

Automatically resolves deadlocks.

Allows authoring of C# 5 friendly async Task methods that are then adapted to JoinableTask.

Less scheduling overhead than IVsTask.

Bidirectional interop with IVsTask (produce and consume them naturally).

Cons:

More overhead than TPL Task.

Not available to call from native code.

To sum up, if you’re writing managed code, any async or scheduled work should probably be written using C# 5 async methods, with JoinableTaskFactory.RunAsync as the root caller. This maximizes the benefit of reusable async code while mitigating deadlocks.

Summary

The JoinableTaskFactory makes asynchronous code easier to write and call for code that runs within Visual Studio. Switching to the UI thread should always be done using JoinableTaskFactory.SwitchToMainThreadAsync(). The JoinableTaskFactory should be obtained from ThreadHelper.JoinableTaskFactory.

As you use these patterns, we’d like to hear your feedback. Please add comments on this post.

Appendix

It is not always obvious whether some VS API you’re calling may require marshaling to the UI thread. Here is a list of types, members, or tips to help identify when you should explicitly switch to the UI thread yourself using SwitchToMainThreadAsync() before calling into VS:

Package.GetGlobalService()

Casting an object to any other type (object to IVsHierarchy for example). If the object being cast is a native COM object, the cast may incur a call to IUnknown.QueryInterface, and will most likely require the UI thread.

Almost any call to an IVs* interface.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/07/asynchronous-and-multithreaded-programming-within-vs-using-the-joinabletaskfactory/feed/6https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/07/asynchronous-and-multithreaded-programming-within-vs-using-the-joinabletaskfactory/Recommended patterns for CancellationTokenhttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/pJJf3ensRZU/
https://blogs.msdn.microsoft.com/andrewarnottms/2014/03/19/recommended-patterns-for-cancellationtoken/#commentsWed, 19 Mar 2014 14:26:00 +0000https://blogs.msdn.microsoft.com/andrewarnottms/2014/03/19/recommended-patterns-for-cancellationtoken/Read more]]>Whether you’re doing async work or not, accepting a CancellationToken as a parameter to your method is a great pattern for allowing your caller to express lost interest in the result.

Know when you’ve passed the point of no cancellation. Don’t cancel if you’ve already incurred side-effects that your method isn’t prepared to revert on the way out that would leave you in an inconsistent state. So if you’ve done some work, and have a lot more to do, and the token is cancelled, you must only cancel when and if you can do so leaving objects in a valid state. This may mean that you have to finish the large amount of work, or undo all your previous work (i.e. revert the side-effects), or find a convenient place that you can stop halfway through but in a valid condition, before then throwing OperationCanceledException. In other words, the caller must be able to recover to a known consistent state after cancelling your work, or realize that cancellation was not responded to and that the caller then must decide whether to accept the work, or revert its successful completion on its own.

Propagate your CancellationToken to all the methods you call that accept one, except after the “point of no cancellation” referred to in the previous point. In fact if your method mostly orchestrates calls to other methods that themselves take CancellationTokens, you may find that you don’t personally have to call CancellationToken.ThrowIfCancellationRequested() at all, since the async methods you’re calling will generally do it for you.

Don’t throw OperationCanceledException after you’ve completed the work, just because the token was signaled. Return a successful result and let the caller decide what to do next. The caller can’t assume you’re cancellable at a given point anyway so they have to be prepared for a successful result even upon cancellation.

Consider not checking the token at all if your work is very quick, or you propagate it to the methods you call. That said, calling CancellationToken.ThrowIfCancellationToken() is pretty lightweight so don’t think too hard about this one unless you see it on perf traces.

Check CancellationToken.CanBeCanceled when you can do your work more efficiently if you can assume you’ll never be canceled. CanBeCanceled returns false for CancellationToken.None, and in the future possibly for other cases as well.

Optional CancellationToken parameter

If you want to accept CancellationToken but want to make it optional, you can do so with syntax such as this:

public Task SomethingExpensiveAsync(CancellationToken cancellationToken = default(CancellationToken)){ // don’t worry about NullReferenceException if the // caller omitted the argument because it’s a struct. cancellationToken.ThrowIfCancellationRequested();}

It’s a good idea to only make your CancellationToken parameters optional in your public API (if you have one) and leave them as required parameters everywhere else. This really helps to ensure that you intentionally propagate your CancellationTokens through all the methods you call (#2 above). But of course remember to switch to passing CancellationToken.None once you pass the point of no cancellation.

It’s also a good API pattern to keep your CancellationToken as the last parameter your method accepts. This fits nicely with optional parameters anyway since they have to show up after any required parameters.

Handling cancellation exceptions

If you’ve experienced cancellation before, you’ve probably noticed a couple of types of these exceptions: TaskCanceledException and OperationCanceledException. TaskCanceledException derives from OperationCanceledException. That means when writing your catch blocks that deal with the fallout of a canceled operation, you should catch OperationCanceledException. If you catch TaskCanceledException you may let certain cancellation occurrences slip through your catch blocks (and possibly crash your app).

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2014/03/19/recommended-patterns-for-cancellationtoken/feed/3https://blogs.msdn.microsoft.com/andrewarnottms/2014/03/19/recommended-patterns-for-cancellationtoken/Immutable Object Graph updateshttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/olqykAyB5Nc/
https://blogs.msdn.microsoft.com/andrewarnottms/2013/05/07/immutable-object-graph-updates/#commentsTue, 07 May 2013 08:31:16 +0000https://blogs.msdn.microsoft.com/andrewarnottms/2013/05/07/immutable-object-graph-updates/Read more]]>In my last post, I introduced a T4 template that constructs efficient and quite capable immutable objects based on the simplest mutable type definition. I also mentioned that the published sample is (necessarily, ala T4 style) open source and hosted online. Two outsiders have already submitted pull requests that have been accepted. Some thoughtful feedback came in the form of comments to my last blog post as well. Thank you for your participation!

The latest incarnation of the T4 template is substantially improved over what my last post described. Here are a few of them:

The immutable types now look and feel more like the immutable collections, having static factory Create() methods instead of a static Default property.

The With(…) multiple parameter method is easier to use.

Partial methods give you the opportunity to set defaults for any property on your type.

Certain fields can be marked as required, so that they are not optional parameters in the Create() method.

Better support for immutable collections as members of the immutable types.

The T4 template is broken up into several files, modeled as extensions, making it easier for you to add additional functionality to the template without having to fork the code in your own codebase, making future updates from the official project easier.

Better support for a network of immutable objects referencing each other (the graph part of the name).

#7 is where I expect to spend much of the future work on this effort.

If this topic interests you, please take a fresh look at the latest version, and keep the feedback coming.

]]>https://blogs.msdn.microsoft.com/andrewarnottms/2013/05/07/immutable-object-graph-updates/feed/2https://blogs.msdn.microsoft.com/andrewarnottms/2013/05/07/immutable-object-graph-updates/Simple immutable objectshttp://feedproxy.google.com/~r/AndrewArnottMSDN/~3/jnhxY0x7v98/
https://blogs.msdn.microsoft.com/andrewarnottms/2013/01/08/simple-immutable-objects/#commentsTue, 08 Jan 2013 12:19:00 +0000https://blogs.msdn.microsoft.com/andrewarnottms/2013/01/08/simple-immutable-objects/Read more]]>We’re all familiar with immutable collections now. But immutability is only as immutable as it is deep. And an immutable collection of mutable objects may not provide the depth you’re looking for. So how can one create an immutable object?

Now that’s fine for very simple objects. But once objects have several fields on them, the code you have to write to create a new object with just one or two fields changed gets tedious to write and maintain. So we can add a helper method for each property, that returns a new object with just that property changed. The method name convention is WithPropertyName.

We can make a couple more enhancements though. For classes with many properties, if you need to change several properties at once, allocating a new object with each property change as an intermediate step is wasteful and can contribute to GC pressure. So we also add a With method that takes optional parameters for every property found on the class, allowing bulk property changes. Finally, for scenarios where you have several changes to make to the object but wish to do so in a multistep fashion (or just prefer property setters to With- method calls), we can construct a Builder class that allows mutation, and then can return the immutable copy when you’re done. This pattern is very similar to String and StringBuilder in the .NET Framework, and also like the recent immutable collections mentioned earlier.

Another enhancement we can make is to make all constructors private. Why? Because a public constructor will end up allowing callers to allocate many copies of the default instance. Since these are immutable, this is just wasted memory and GC pressure. Alternatively, a static Default property that returns what a public default constructor would have but sharing the instance with all other callers, alleviates the pressure while also putting the caller in the “mood” for functional programming.

What does the resulting immutable object look like? Considering it’s just a simple immutable object with one property, it’s rather large.

Of course adding more properties will be common, and the code will increase a bit more with each one. Since each added property means two new fields (one for the immutable type and one for the Builder), two new properties, and one new method and several other methods to update, this gets quite tedious to write and maintain.

So what can we do to simplify immutable programming? After all, immutable programming is supposed to be easy, right? Why can’t authoring immutable types in C# be as easy as mutable types (or immutable types in F#)? There is a better way, using Visual Studio T4 templates.

Consider this simple definition of a mutable type:

class Fruit {
string color;
}

We can leverage code generation to produce an immutable version of this class that fulfills all the patterns described above. And due to the built-in support for T4 in Visual Studio, the experience can be just as easy as maintaining the above simple mutable type. Adding a property to the Fruit class is as simple as adding one line for the field in the above mutable type. Saving the updated file will immediately update the code generated immutable Fruit class, allowing you to get Intellisense and compile against the immutable version. T4 is built into Visual Studio, so it will already work for you today. Because code generation happens at design-time, you don’t need any additional support for it in any customized build system you may have.

I’ve published the code generation tool in source form as a sample. You can see a live example of the before and after code generation. The README in that project describes more and provides instructions for incorporating this tool into your source projects.