Async tasks in C++

The introduction of the concurrent memory model and the concurrency library in STL are major features brought by C++11. With prior versions, we had to resort to platform-specific extensions or library in order to write multithreaded applications, limiting the portability of the code.

One of the new facility introduced is std::async(), which allows to make use of concurrency at an higher level of abstraction. It relies on « tasks » instead of manually creating and managing threads. But it also comes with some serious caveats that every programmer should be aware of.

Introduction to tasks

Thread based parallelism

C++03 lacks any notion of thread management. Thus, in order to create multithreaded applications, programmers often relies on C APIs, such as POSIX or Win32, and have to be familiar with the concept of threads. But manipulating threads can quickly become a burden. You have to manually launch and join them, and be careful that you don’t exceed the capabilities of your hardware and OS, as threads are costly.

Such a traditional thread management is now provided by « modern »C++, around the class std::thread.

Task based parallelism

C++11 also introduced the notion of task based parallelism.You define jobs (aka tasks) that can and may be run concurrently, and you rely on the system to spawn threads, depending on the available resources available. Unlike with manual thread management, creating tasks is supposed to be burden free for the programmer and always optimal. In order to fulfil these pledges, the underlying has to be quite smart. It usually relies on global thread pools. The tasks themselves may have to be migrated from one thread to another. Or, if a task is blocked waiting for a resource, it may be taken off its thread so another can replace it.

Tasks in C++

std::async

In order to work with tasks, C++11 introduced the function template std::async. Its purpose is to launch the provided task on a thread, which may be a new thread, a recycled thread or the current thread. The sophistication behind the scene can greatly vary from one implementation to the other.

std::async takes as parameters, a launch policy (more on that later) and a function object (along with its parameters) forming the « task »to be processed. std::async returns a std::future.

std::future

std::future hosts the result promised by a task. This result is accessed by the get() operation. If the task has already completed its job, it may already be available. Otherwise, get() launches the task in the current thread and blocks until the result becomes available.
If an exception is thrown by the task, it is propagated to the current context. Thus, error handling is greatly simplified!

Limitations

Unfortunately, the way C++ standardized task based parallelism is far from perfect, and there are many limitations every programmer should be aware of before using tasks!

The first, and most important limitation is that, because of C++11 itself, it is extremely difficult for the STL implementers to provide a true task based parallelism!!!

A true task, should not be too closely tied to its underlying thread. It should be easy to reuse threads for multiple tasks, and to move them to another thread or unload them when being blocked.
But C++11 introduced new concepts such as thread_local. A thread_local variable is some kind of global variable that’s only visible from one thread and should be initialized and destroyed along with it. As you can imagine, difficulties arise if a task making use of thread_locals had to be moved to another thread…

Many other problematic scenarios could also appear due to locks and mutexes.

The second limitation is that the C++ standard does not define precisely enough what to expect when using std::async to create tasks. When you use std ::async, you can specify a launch policy.

std::launch::deferred will force the task to run in the same thread. std::launch::async tells the underlying that the task has to run concurrently. And if you don’t precise a launch policy, a default one is chosen: std::launch::deferred | std::launch::async. It means that the library will decide which policy is to be applied, according to its own criteria. That’s the one you would expect to use if you’d want to profit from task based parallelism as I presented it earlier. The problem is that the standard imposes nothing concerning those criteria! Thus, the behavior of a task created with the std::launch::async may differ greatly from one STL implementation to the other.

For instance:

GCC 4.8 :

std::launch::deferred | std::launch::async will always launch the task as deferred

std::launch::async will create one thread per task

Visual C++ 2017

std::launch::deferred | std::launch::async will make a decision according to the resources

std::launch::async is said to use a global thread pool

It is clear that this is problematic if you aim to write portable code! At the extreme, if you spawn too much tasks, it will behave accordingly to expectations if compiled with Visual, but crash if compiled with GCC4.8, due to resource exhaustion!

Note: I am aware that more recent version of GCC are closer in sophistication to Visual 2017. I just wanted to illustrate my point with two extremes.

For all those reasons, it is safer to consider that std::async do not provide a true task based parallelism!

Conclusion

Despite all the limitations discussed above, it is highly advised to make use of std::aync and futures whenever possible ! Indeed, threads can be sensitive and have a tendency to terminate() if not handled with care. Furthermore, having an easy interface providing futures is worth the limitations.

The only thing is to remember that std::async do not provide a true task based parallelism!
If you need to run your tasks concurrently, it is suggested to use std::async explicitly asking for a std::launch::async policy and consider that each task will run in its own thread.
Also, if you want to stay away from troubles, don’t use std::async tasks to perform I/O operations, or manipulate mutexes. That last part is not from me, but from Bjarne Stroustrup himself!

If you follow these advices, you will benefit of all the advantages brought by std::async and std::futures over without any risk.

It is only if you need more control over your « tasks », that you will have to fall back to manipulate plain std::threads.

After all this is what C++ is all about: giving freedom of choice the programmers. It’s just that sometimes freedom can become tricky 😉

Related Posts

My port of Conway’s Game of Life to the Apple II is now complete! The whole code is available on GitHub. Plus, there are four posts on this documenting my progression that could interest those Read more…

The Apple II was innovative in many ways. One of its most remarkable feats was that it was the very first home computer supporting color graphics! Think about it! While all its competitor could display Read more…