Throughput and Concurrency: Reducing Load Testing Time and Costs – Part 2

July 14, 2011

In my previous post I discussed that when used properly, throughput and concurrency tests can drastically reduce testing time and costs. It covered why it’s critical to understand the difference between these two types of tests, and how they can be used to effectively and efficiently test the performance of your web application.

In this post, I’m going to cover how to apply “Think Time” to load testing as an effective way of simulating real user behavior, but it is also an effective means for controlling the length of a user session.

Importance of Think Time in Your Load Testing

At the very core, the only difference between a throughput test and a concurrency test is the length of the user sessions. In our Web Load Testing solution, session length is controlled by the “think time” within a script.

“Think Time” is a time delay that is added into test scripts to simulate the pacing of real users using a web application.

Try to imagine how a real person behaves when visiting a website or page when thinking about “think time”. Most users are going to pause and scan the content that has just been displayed after a page has finished loading in the web browser and before they decide to click on a link or leave the site.

Let’s say the page contains a form: Real users will typically require some time to fill in that form before they can submit it. This user processing time between page requests is what “think time” tries to emulate.

Adding “think time” to a load test is not only an effective way of simulating real user behavior, but it is also an effective means for controlling the length of a user session. The longer the session length the greater the number of concurrent user sessions will be on the system.

Comparing Examples of Load Testing

To take this concept a little further, let’s take a look at the following example. Suppose we have a script (or transaction) that includes 3 pages, and each page takes approximately 2 seconds to load and 6 seconds for the entire transaction to complete.

In theory, with only one virtual user, we could complete two transaction runs within 12 seconds. Now, let’s suppose that we decide to add a 2 second “think time” after each page.

A transaction will take 12 seconds to complete for one virtual user when a 2 second think time is added to the script. If we wanted to achieve the same throughput as the script that has no “think time”, we would need twice as many virtual users in order to do it.

A test with no or very little think time would be considered a throughput test. By adding “think time” to the test, you can maximize the number of concurrent users on the application, while achieving the same throughput. A test with significant “think time” added to the test can be considered a concurrency test.

What Type of Load Test Should You Use?

After seeing the effect of “think time” on a script and a test, you may be asking yourself, why even use throughput tests? With a concurrency test you can achieve the same throughput, and the test will be more reflective of a real user experience with “think time”. This is true, and if time and money were unlimited, we probably wouldn’t need throughput tests.

However, time and money is often limited. Throughput tests are an effective means for reducing the cost and time of load testing. Looking back at our “think time” example, with a throughput test we can test the throughput of a web application with less virtual users than a concurrency test. Using less virtual users can shorten the time required to test.

If throughput tests are more cost effective and require less time to conduct and concurrency tests are more realistic and effectively test the concurrency of our application, what type of tests should we use?

The simple answer is that you should use both.

We recommend that you conduct your initial testing using throughput test to find the “low hanging” bottlenecks within your application, then start testing multi-script scenarios in concurrency tests to test your application using load that closer reflects real user traffic.