A benchmark of performance testing costs

Capacitas working in partnership with a world class academic institution

For several years now Capacitas has been trusted by one of the world’s most prestigious universities to provide performance assurance services for its IT systems. This is particularly critical when major IT programmes are being launched, because without performance assurance the programmes will not deliver the improvements required. In the worst case, services may not work at all. Recently, Capacitas has delivered performance assurance for a number of key development programmes. This work primarily involved the Risk Modelling and Integrated Load Testing elements of the Software Performance Engineering service.

The Customer originally planned to build an internal load and performance testing capability. However, they accepted that they would need external assistance, as they had no performance engineering skills available. Capacitas were therefore engaged, to help them mature into this function.

Variable performance testing demand

From the outset of the relationship, it was clear that consultant demand would be driven by the seasonal nature of client project delivery. The level of activities varied - sometimes multiple concurrent performance testing projects were active, at other times there were periods of low activity. Generally, peak performance testing activity was biased towards the late summer, at the start of the new academic year.

There were firm deadlines, by which the software had to be delivered, and performing well.

How did we deliver staffing for test demand cost effectively?

An analysis of time recording was done for Capacitas consultants working on load and performance testing projects. The results clearly showed that the peak consultant commitment in August, September, and October was five times higher than the normal level of Consultant commitment outside of the peak period. This shows how Capacitas flexibly scaled up their consultant resources.

On a like for like basis, Capacitas was clearly the most cost effective option compared to both contractor and permanent options. Capacitas resourcing flexing approach is much more cost-effective, as both alternate options these resources would have had to be active for the quieter periods. This clearly demonstrates the value and the flexibility of our approach. The methodology and underlying assumptions used in the cost comparison were fully verified by the customer.

In Conclusion

We successfully delivered major change, to the deadlines, with costs firmly under control.

It is possible to get a quart into a pint pot! The Capacitas resource model supports small and large customer accounts alike, using the team of highly skilled consultants flexibly. Often for a lower cost than using your own staff, or contractors.