Testing

Unit Testing with Python

To skip a test routine conditionally, use the class methods unittest.skipIf() and unittest.skipUnless() as in Listing Twelve:

Listing Twelve.

@unittest.skipIf(self.fooC > 456, "Skip over this routine")
def testB():
"""Test routine B"""
# the following code will run only when fooC is less
# than or equal to 456
fooA = narf()
self.assertNotEqual(fooA, 123)
@unittest.skipUnless(self.fooC > 456, "Skip over this routine")
def testC():
"""Test routine C"""
# the following code will run only when fooC is
# greater than 456
fooA = zort()
self.assertNotEqual(fooA, 123)

The first method causes a skip when a condition is met, the second when a condition is not met. Both get two arguments: the condition and the reason for the skip. Both must be placed before the test routine. The condition may involve a class property or another class method.

To fail a test routine, use the instance method fail() as in Listing Thirteen:

This method takes a log message explaining the reason for failure. Like skipTest(), the fail() method goes inside the test routine. Code placed after the call to fail() will not run, but test routines after the failed routine still get to run.

Viewing the Test Results

There are two possible forms of output from the TextTestRunner: console text or a TestResult object. First, let's look at the console output, which shows each test's result. You can control this output by passing three optional arguments to the class constructor.

The first argument (labelled stream) sets the output destination. This defaults to sys.stderr if one is not specified. Next argument (labelled descriptions) controls how errors and failures are reported. Passing a True (default) tells the runner to name those routines that erred, failed, or skipped. Passing a False tells it not to.

The last constructor argument (labelled verbosity) sets the level of detail. There are three possible levels. For a verbosity of 0, the results show only the number of executed tests and the final outcome of those tests. For a verbosity of 1, the results marks a successful test with a dot, a failed one with an F, a skipped one with an s, and an erroneous one with an E. And for a verbosity of 2, the results lists each test case and test routine, plus the outcome of each routine.

Note that the sFEF line is replaced by four pairs of lines, each pair naming the test case and routine, the final test state, the docstr comment of each routine, and any assert messages.

The second type of output is the TestResults object. This one is returned by the runner object after all the tests have run:

fooResult = fooRunner.run(fooSuite)

The TestResult object has two sets of property accessors (Figure 6).

Figure 6: The TestResult object (in part).

This is not a complete set of accessors, but these are ones you will most likely use. The first set returns a list of tuples. Each tuple reveals how each test routine fared when executed. The errors accessor lists identified routines that raised an exception. Each tuple has the name of the test case and routine, the location of the test script, the line position of the error, a trace-back, and a reason for the error.

The failures accessor lists test routines that failed. Its tuples contain the same information as tuples from the errors accessor. The skipped accessor lists routines that were skipped, conditionally or not. Its tuples name the test case and routine, and give the reason for the skip.

The second set of accessors provides additional data. testsRun gives the number of test routines that ran, regardless of outcome. And wasSuccessful() returns a True if all routines ran without problems, False if at least one routine had a problem. Notice this last accessor is written as a function.

This script uses the same FooTest case defined in Listing Fourteen. After it invokes the run() method in the runner object fooRunner, the script stores the results into the local fooResults. Then it invokes each accessor and prints the test result on the console window.

The second line summarizes the results. It shows the total number of test routines and how many routines have erred or failed. The paragraph below it reveals the erroneous routine: test_C(). It also reveals the cause of the error: the undefined variable fooB.

The next clump of text reveals the failed routines: test_B() and test_D(). It reveals why test_B() failed: two unequal values passed to assertEqual(). And it reveals that test_D() has explicitly called the instance method fail().

Below that is shown the skipped routine, test_A(). The next line shows what problems were encountered. And the last two lines report that a total of four test routines ran, confirming what was reported initially.

Conclusion

Python has substantial resources to enable unit testing. In this article, I looked into the unittest module and examined those classes essential for unit testing. I showed how to create a test case, how to design the test routines, and how to gather several test cases into one test suite.

The unittest module can do more than just run basic tests. With it, we can design tests that focus on exceptions or tests that do pattern matching. We can load and execute tests stored in separate files. We can even design tests that use simulated objects (known as mocks). But these are topics for a later article. Meanwhile, the "Recommended References" can provide additional guidance on these more-advanced features.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!