Comparison of Unit-Level Automated Test Generation Tools,

Shuang Wang and Jeff Offutt.

Abstract

Data from projects worldwide show that many software
projects fail and most are completed late or over budget.
Unit testing is a simple but effective technique to improve
software in terms of quality, flexibility, and time-to-market.
A key idea of unit testing is that each piece of code needs
its own tests and the best person to design those tests is
the developer who wrote the software. However, generating
tests for each unit by hand is very expensive, possibly prohibitively
so. Automatic test data generation is essential to
support unit testing and as unit testing is achieving more attention,
developers are using automated unit test data generation
tools more often. However, developers have very
little information about which tools are effective. This experiment
compared three well-known public-accessible unit
test data generation tools, JCrasher, TestGen4j, and JUB.
We applied them to Java classes and evaluated them based
on their mutation scores. As a comparison, we created two
additional sets of tests for each class. One test set contained
random values and the other contained values to satisfy
edge coverage. Results showed that the automatic test
data generation tools generated tests with almost the same
mutation scores as the random tests.