Parallel

Reducing Computing Time with Multithreading

By Daniele Bochicchio, Stefano Mostarda, and Marco De Sanctis, June 17, 2010

Multithreading can have a big impact on application response time

How To Increase Performance with Multithreading

Applying multithreading techniques is useful when you have to deal with multiple requests and you do not want to slow down the process while waiting for results. The typical example is a system made of many requests to different web services. If you need to implement something similar, you will probably end up using a simple for iteration and calling each web service in this block.

This technique may work with few requests but to speed up execution you need to use multithreading.

Problem

Let's suppose we have to gather some data using different web services and display the results on the page, just like a flight-comparing engine would do.

We want to avoid latency and provide a better experience for the user while waiting for different external services to respond to our requests. Normally, if we opt to execute a single task at a time, the total time will be the sum of the entire operation. We can dramatically improve the speed by executing the tasks in parallel -- we will gain in total response time.

Solution

In heavy-load scenarios where you need to execute different tasks at the same time using a worker thread, you may reduce the total execution time. A worker thread is a secondary thread created by the primary one to accomplish a specific task.

The .NET Framework has a specific namespace, called System.Threading, to support threads and a specific class, named Thread, to represent the concept of a thread in managed code.

Thread has a special constructor that receives the code to be executed and a Start method to begin execution. When the thread is created, there is a fork in the execution flow: the primary thread will continue its normal execution, while the secondary will start its work. To provide a true multithreading experience, we are going to execute every request as a separate thread. Using this approach, the total time for the request to be executed is not the amount of different requests but the maximum one (plus the overhead of creating, destroying, and joining threads).

Even if possible, it is not a good idea to directly instantiate threads. For such a specific scenario, a specialized class, called ThreadPool, is used. This class represents a pool of threads managed by the CLR itself and can be used to coordinate them. When using a technique like this one, we need thread synchronization. Each call to the QueueUserWorkItem method will immediately return, so we need a way to notify our class that each thread has completed its work and that the results are ready to show. To accomplish this task, we need to manually use a WaitHandle class, shown in Figure 2.

Figure 2: Threads generation and synchronization need to be handheld to work with threads properly. When completed, a single thread will notify the ThreadPool.

The problem at this point is that, while accessing the List to add our results, there is no guarantee that there will not be collisions from different threads trying to modify the same collection at the same time. List is in fact not thread safe, so we need to synchronize the modifications using the lock keyword (in C#) or the Monitor class. The whole code is in Listing 1

In Listing 1, you can find the general structure of the engine. Its inner working is contained in the code in Listing 2, where each result is retrieved from different providers and added to the results collection using a thread-safe approach.

The IFlightPriceProvider interface is guaranteeing that every provider has the GetFlightPrice() method to load results and is part of our design strategy, often referred to as Provider Model. The providers attached to this example are just for test and, in order to simulate latency, they have a call to Thread.Sleep to freeze execution for a couple of seconds. A simple implementation is in Listing 3.

In real-life scenarios, you will insert your code in this method and populate a new instance of FlighPriceResult class to return the corresponding flight price. To effectively start the work, we need to create a page with a Textbox to enter the flight number and a Button to execute the code, as shown in Listing 4.

The code in Listing 4 is very simple; the engine will start, saving its instance in Session so we can access it later. The results.aspx page includes a code to check in Session for the instance of PriceEngine class that originates the threads, checking at intervals for the execution to be completed. By using a simple reload of the page, as in Listing 5, we can check the state of the job and, in case it is done, just display the results to the user.

The sequence of the entire workflow is displayed in Figure 3, and it is very similar to what comparing price engines are doing to speed up their execution, while you are waiting for the results to come in the web page.

Figure 3: The sequence of our multithreading system. First of all, the providers are initialized and their work executed. When all the providers are done, the engine is notified so the results may be displayed.

Dealing with multithreading is not simple because, as previously shown, you have to take care of the details; just be sure to use thread-safe collections. In any case, the results are very interesting. When dealing with a solution like the one exposed here, the performance boost you can achieve by simply letting different threads simultaneously execute different tasks is visible at first glance by both you and your customers.

Multithreading can boost specific applications, like the one in this scenario, by leveraging spanning the work on different threads. It is not intended to be used by every application because there are cons to be considered. ASP.NET uses a thread pool and it is useful to remark that the pool size includes both ASP.NET-generated threads and the ones generated in your code. Remember that this technique, in very high traffic applications, can lead to consuming the entire pool size, and you'll need to increase it accordingly if you notice that its size is very low. Just for your information, current threads can be monitored in Control Panel under Performance Monitor. Using threads to execute multiple tasks separately is a good design decision to increase performance dramatically and to leverage ASP.NET multithread nature in a very good way.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!