Libtap issued a diagnostic to say that you planned too many tests. In this case the diagnostic was correct, but you could also get this message if some spectacular failure makes the program exit cleanly before all the tests have run. In addition, it printed failure--meaning that the test program returned nonzero, indicating a failure. This is the handiwork of the exit_status() function, and makes it possible to simply run the test and ignore the output to get a ok or not ok (although this would not catch the spectacular failing case here).

Responsibility of Tests

Unit tests have several responsibilities. Above all, they should detect when something has gone wrong. In order to help you quickly and easily find the problem, they should also attempt to give good clues as to what has gone wrong and where the problem might be.

I once needed to test an AI for an Othello (aka Reversi) game. To make sure that the AI picked the expected move at each point, I compared each state in a run with the states of a known good run. Simply doing square-by-square comparisons over the entire grid at each state would have detected errors, but it would have made it hard to see in which situation the errors occurred. Actually creating (not to mention updating) the tests would have been a nightmare also.

Instead I chose to code the test in two parts. The first was a very simple C program to search for and apply moves and print out the current state at each step. This part resembled:

Notice that this last test did not actually use libtap at all. Instead, it had a C component and a Perl component. Using Inline::C, you could improve on that and put both the C and Perl components in the same file. Note that using two separate parts has the benefit that if Perl is not available the C component can still run, although you must manually compare its results with the canned run in the Perl component.

Availability

Both Test::More and Test::Harness are part of the core in recent Perl distributions. Libtap is unfortunately not (yet) a commonly installed library. However, it has a liberal license and consists of only two source files, so if you want to make sure that your users can run your test suite, you can bundle libtap with your software.

Conclusion

This article explores various ways of using Perl to help testing your C code. First, it briefly examines the Test Anything Protocol, the heart of Perl's well-established test framework. It then shows how libtap can help you produce TAP output from C programs. It rounds off by using Perl to test programs by comparing their output with known good reference output.

Hopefully, this should give you some ideas for improving your own testing.