A Balancing Act or how to test software most effectively within the available time

Is it possible to create a perfect information system? It should be, given infinite manpower, money and time. In real life, however, every system development project has to be executed within severe constraints on all of these. Development teams and leaders work on doing the best that they can with the available resources to meet deadlines.

At every stage in the software development process, compromises have to be made in order to ensure that the solution is ready in time. Even the smallest delay would result in cost escalations, which directly impact the profitability of the whole endeavor. The testing phase, which accounts for about 30% of the total time required to execute any project, is also subject to these constraints.

Especially in a complex system with many possible combinations of steps (known as paths in testing parlance) involved in processing information, it would take forever to test everything. The challenge for the tester, therefore, is to ensure that a balance is struck between the time taken and the thoroughness of the tests. There are several ways in which this problem is addressed.

One way is to test a fraction of all the possible paths. Testing is stopped once a predetermined percentage of possibilities has been addressed. Another way is to test only the most important or critical of the possible combinations. Yet another way is to run the tests for a fixed number of hours or days.

Irrespective of the method chosen, the end result is that all of the defects do not get detected and rectified. Apart from the method chosen to restrict the extent of testing, the quality of the tests themselves also reflect on the effectiveness of the tests.

Sukriti Goel, a research scholar at the IITB-Monash Research Academy in Mumbai, has been working to find ways to maximize the effectiveness of testing within the usual constraints. Sukriti’s research has centered around two objectives.

The first objective was to minimize the number of tests required to detect at least the same number of defects as found using existing methods. By reducing the number of tests, the available time can be used for testing more of the possible paths. The second is to find ways to automate the process for creating tests of maximum effectiveness.

Both of the above will be instrumental in significantly reducing the time required for testing, which is traditionally about 30% if the total development time. Apart from initial testing, these would also reflect on the time taken to test the fixes to the defects that are detected. This would enable the quicker, more cost-effective and efficient implementation of business process automation solutions.

Given the benefits that it brings to business and industry, the work done by Sukriti is typical of the ethos of the IITB-Monash Research Academy. Says Prof. Mohan Krishnamoorthy, the CEO of the Academy, “I am excited about the IITB-Monash Research Academy and its potential to make significant contributions to business and society in India and Australia. I am confident that the unique collaboration model that the ‘Academy’ follows will attract innovation-led industry, top-ranking students and the best researchers to work together to solve grand-challenge problems that need to be solved.”

IITB-Monash Research Academy is a Joint Venture between IIT Bombay and Monash University. Research scholars study for a dually-badged PhD from both institutions, and enrich their research and build collaborative relationships by spending time in Australia and India over the course of their degree. Established in 2008, IITB-Monash Research Academy aims to enhance scientific collaborations between Australia and India.