Challenges in Software Testing

Software Testing & the Iron Triangle

The iron triangle is common to many software testing managers. It’s that model of constraint where quality, time and cost are shown on the corners of the triangle to demonstrate tension between the three aspirations within product delivery. Its a recognition that in product or system delivery attempt at optimisation of one these attribute will impact the others.

For example in test management, its common to think that;

If we develop a product to a high standard and quickly, it wont be cheap (for example, we will need extra testers to complete the project on time).

If we develop a product quickly and cheaply, the quality will be low (for example, we will have insufficient time to test properly).

If we develop a product a high quality and cheaply, it will take a long time (for example, reducing testers will increase the time it takes to test).

I’ve known test managers to think like this, and this used to be my thinking too.

I’ve come to realise though, that in order for this model to hold true, we need to allow for some assumptions.

For example, in testing we need to assume:

All testers are the same when it comes to testing. Skill is inconsequential.

Testers don’t take risk into account when testing.

Valuable testing cannot be achieved without an impact on time or cost.

Efficient testing comes at a cost to the quality of testing.

Its possible, that by examining & perhaps exploiting these assumptions, we can can come up with an approach to testing that is cost effective, timely AND can deliver quality testing.

Skilled Testers

If you’ve spent a little time around testers, you come to the realisation that not all testers are alike. Some testers are great at finding bugs quickly while others seem to only find superficial bugs. Part of this maybe intuition, but if you observe a skilled tester, you will notice that good testers ask great questions, are good at evaluating information, understand how to model a product and are not fearful of technology.

Risk Management

Good testers understand the importance of risk management in testing. Risk Management looks at the two attributes , the severity of failure and the impact of that failure on business. This approach can help make difficult decisions about what cover in your testing. Be careful not to confuse this with Risk Based Testing, which is a test design technique to help you find failure faster. In the hands of a skilled tester, both approaches may help deliver within time and budget and offer valuable testing information.

What is Quality?

Gerald Weinberg defines quality as “value to some person”. Michael Bolton has added, “Value to some person at some point in time”. Quality is a relative term and can mean different things to different people. What quality means over time, also changes. Think of the telephone system. It would be intolerable to us today to have a phone that we were unable to move around it. That would be considered a “low quality” product. However thirty years ago, this was considered state of the art! By understanding what quality means to our stakeholders testers are able to deliver information on that. This is valuable testing. Valuable testing is not necessarily linked to cost or time. Its more about asking the right question and understanding the context in which the testing will be performed.

Software Testing is often seen as a cost that needs to be kept low and reduced. Traditionally, this as led to offshoring where the cost of testers appears to be lower. The downside to this approach is that testers have been distanced from the stakeholders. This makes it a little harder to offer ‘quality’ testing.

Automating the testing execution is another way that teams attempt to reduce cost, but often it ignores the hidden cost of maintenance which is required to keep scripts upto date. Automation of test execution has its place, but it requires understanding of it’s place in the testing approach to make it really effective.

There is another way to view efficiency and that is by reducing waste. There can be large amounts of wastage in testing. You only have to look at the vast amount of rarely read documentation to understand what I mean. To reduce waste requires software teams to better understand what testing means to them. Why do they test? What benefits do they hope to achieve? Is it realistic? These are questions that every company needs to ask themselves when reviewing their testing process. Its only then can you start to examine what testing can do for your company. It is possible to deliver efficient and quality testing within that context.

It is possible to deliver software on time and on budget without compromising on the quality of testing. In fact, its possible to deliver with an improved quality of testing. At Testing Times, we do this everyday. Find out how we can help you get the testing that your product deserves.