Abstract

Software testing is an essential process in software development. Software testing is very costly, often consuming half the financial resources assigned to a project. The most laborious part of software testing is the generation of test-data. Currently, this process is principally a manual process. Hence, the automation of test-data generation can significantly cut the total cost of software testing and the software development cycle in general. A number of automated test-data generation approaches have already been explored. This paper highlights the goal-oriented approach as a promising approach to devise automated test-data generators. A range of optimization techniques can be used within these goal-oriented test-data generators, and their respective characteristics, when applied to these situations remain relatively unexplored. Therefore, in this paper, a comparative study about the effectiveness of the most commonly used optimization techniques is conducted.

Keywords

Appendix

In this section we provide a skeletal outline of the programs used in this study. These outlines provide all of the decision information utilized by the algorithms, but have the sequential code deleted for the sake of brevity. In addition, a brief discussion of the translation of the program into goal-oriented optimization criteria is presented for each program.

1. Hex_dec conversion

... ...

while ((c=getchar())!=‘\n’) //1

... ...

if(c >=‘0’ && c <=‘9’||c >=‘a’ && c<=‘f’||c>=‘A’ && c<=‘F’) //2

... ...

else

... ...

if (i<=MAX) //3

... ...

else printf(“\nMaximum 7 digits of hex number”);

... ...

for (j=0;s[j]!=‘\n’; j++) //4

if (s[j]>=‘0’ && s[j]<=‘9’) //5

... ...

if (s[j]>=‘a’ && s[j]<=‘f’) //6

... ...

if (s[j]>=‘A’ && s[j]<=‘F’) //7

... ...

... ...

There are 7 decisions in Hex_dec. 36 test requirements need to be satisfied to obtain complete condition-decision coverage. Thus, a maximum of 36 objective functions are generated to allow the test generators to calculate the value of objective function \({\Im }\)(x). For example, consider the following fragment of code:

if(c>=‘0’ && c<=‘9’||c>=‘a’ && c<=‘f’||c>=‘A’ && c<=‘F’)

{...

}

There are 6 conditions and one decision that need to be evaluated independently. According to the requirement of condition-decision coverage, 14 test requirements should be satisfied. Thus, 14 objective functions are generated as below.

Compared to other programs used in this paper, this program has more functions and more complicated relationship between the input parameters and the variables that appear in the conditions that will be evaluated. Most of the decisions do not include multiple conditions but some nested decisions exist. For example, consider the following fragment:

if (m<JAN || m>DEC) //5

... ...

else if (d>daysInMonth(m, y) || d<1) //6

... ...

else if (y<YEAR1 || y>YEARMAX) //7

... ...

else if (y==YEAR1) //8

{

if (m<OCT) //9

... ...

else if (m==OCT && d<15) //10

... ...

else

In order to obtain a test case to satisfy the decision if (m<OCT), the test case also need to satisfy the condition if (m<JAN||m>DEC) and (y==YEAR1), and not satisfy (d>daysInMonth(m, y)||d<1) and (y<YEAR1||y>YEARMAX). Since the test data generators discussed in this paper does not involve any static analysis of the source code, the test data generators only rely on the instrumented code to guide the search. The generating of objective function of the decision if (m<OCT) is still straightforward. To take the “true” outcome of the decision if (m<OCT), the objective function is generated as below:

where oct=10, and p is a significant value 2147483647.There are in total 84 test requirements that require to be satisfied to obtain full condition-decision coverage in the Timeshuttle program.

3. Perfect number program

... ...

if ((num % 7)==0) //1

... ...

if ((num % 11)==0) //2

... ...

if ((num % 13)==0) //3

... ...

else

... ...

switch(ok)

{

case 1: //4

... ...

case 2: //5

... ...

case 3: //6

... ...

default:

... ...

}

void myint::sum()

{

... ...

if (temp==0) //7

... ...

else

... ...

}

void myint::square()

{

... ...

}

void myint::prime()

{

for (i=1; i<(0.5 * num); i++) //8

{

if (i !=num && num !=1) //9

{

if ((num % i)==0) //10

... ...

}

else

;

}

if (ok>0) //11

... ...

else

... ...

}

void myint::perfect()

{

for (i=2; i<(0.5 * num); i++) //12

{

if (num % i==0) //13

... ...

}

if (perfectsum==num) //14

if num>100 //15

if num>1000 //16

... ...

else

... ...

}

int main()

{

... ...

}

There are 37 test requirements that should be satisfied to obtain complete condition-decision coverage of the perfect number program. The objective functions are generated in the similar way as in the Hex_dec and Timeshuttle programs. For example, for decision 16

Execution of the instrumented code provides information about the values of tri, i, j and k allows the calculation of \( \mathfrak{J} \)(x, y, z). Where p is a significant value and it is the maximum range of input space 2147483647 in the triangle program, m is a value between 0 and 1, which is 0.7 in our experiments.

To ensure the condition tri==2 take value ”false”, the following function is built:

There are 16 decision branches (which are identified in bold) in Rescue. According to the definition of condition-decision coverage, the test generators need to generate the test set which satisfy will 46 test requirements to obtain a complete coverage. The program is instrumented with additional code that reports the information needed to the test data generator to calculate the value of objective function \( \mathfrak{J} \)(x). For example, the first branch in Rescue is

if(!(code>9999 && code<100000))

There are two conditions in it, code>9999 and code<100000. To obtain the complete condition-decision coverage, test data must make each condition take the true and false value, and exercise both the true and false branches of the decision. Thus, 6 test requirements need to be satisfied to obtain complete condition-decision coverage. These 6 test requirements and their corresponding objective functions are shown below.

For the fourth branch if(digit4==digit5), the way to construct the objective function is slightly different from the first branch. Two test requirements must be satisfied; the objective functions are shown below.