Fuzz Testing of Application Reliability

Fuzz testing is a simple technique for feeding random input to applications.
While random testing is a time-honored technique, our approach
has three characteristics that, when taken together,
makes it somewhat different from other approaches.

The input is random.
We do not use any model of program behavior, application type, or system
description.
This is sometimes called
black box testing.
In the command-line studies (1990, 1995, and 2006), the random input was simply
random ASCII character streams.
For our X-Window study (1995), Windows NT study (2000), and Mac OS X study (2006),
the random input included cases that had only valid keyboard and mouse
events.

Our reliability criteria is simple: if the application crashes or hangs, it
is considerd to fail the test, otherwise it passes.
Note that the application does not have to respond in a sensible manner to the
input, and it can even quietly exit.

As a result of the first two characteristics, fuzz testing can be automated
to a high degree and results can be compared across applications, operating
systems, and vendors.

If you have reported on the use of the fuzz tools on
testing other systems or more recent testing of the systems that we have
tested, please
send us email
and we would be glad to provide a link to the report: