Introduction

This unit testing framework consists of about 125 lines of code in a single header. It aims to be the simplest way to get started writing unit tests for C++ developers. Being simple, it is also easy to customize. Many unit testing frameworks require linking to a separate library or require jumping through several hoops just to get started. This can make it more difficult to begin writing tests.

One of the situations developers often face when they set out to write unit tests is that they're already working on a project. If the project isn't broken out into independent libraries, it can be difficult to write stand-alone unit tests. In addition, many of the core functions of a program simply can't be broken out into a separate test executable. Ideally, a developer should be able to write a set of unit tests, include it with a single function call within the program itself, and easily #define the test out later on.

This may not seem like a strong approach to software engineering. However, it is generally acknowledged that testing early and often is better than the alternative. By minimizing the effort required to get started writing tests, the goal of testing early can be more easily accomplished. As the test suite grows and the project moves forward, the tests can be factored into a separate library or executable as time permits.

Background

I started my quest by looking for existing solutions. There is a lot of interest and activity around the xUnit frameworks within the unit testing community. It originated with Smalltalk's SUnit framework, which inspired the developers of JUnit. This, in turn, inspired the creators of NUnit, CppUnit, and several other similar frameworks.

One of the things that most of these frameworks have in common is that they are built using languages that support some reflection capability. This makes it easier to assign attributes to a test function or a setup function and have it automatically included in a test run. C++ developers are not so fortunate. C++ developers must do things more manually (or resort to templates or macros). This isn't a big deal, though. It's what we expect.

Many articles have been written on unit testing in the xUnit community and the various libraries available. However, there is surprisingly little written about options for C++ developers. One such article at Games from Within offers a survey of some of the frameworks available.

For an introduction to unit testing in general, have a look at one of the comprehensive articles here at The Code Project.

Design

After evaluating several frameworks, I decided that none of them met my basic criteria of being extremely simple to use and modify. I decided to see how hard it would be to write a framework that would fit into a single header and would consist of as few lines of code as possible. Here is a list of my basic design criteria:

Fits into a single header file (no source modules or libraries)

No longer than a couple of hundred lines of code

Easy to modify and extend

Re-routable message output

Optional macros

No templates

No dynamic memory allocation

Usable in an embedded environment

Usable on down-level C++ compilers (no fancy C++ features)

The design constraint that it should not dynamically allocate any memory would allow tests to be created on the stack. This would make it easy to write a simple main program entry point and just declare and run the tests all at once without worrying about cleanup or memory leaks.

A consequence of these constraints was that a class would be required for each test. The alternative of using a function pointer wouldn't allow for chaining tests within a suite without allocating memory. Also, it didn't seem in the spirit of C++ to use function pointers.

Using the Code

To get a test up and running, three things are necessary:

A test case must be written

A test suite may be written

The test suite and test case must be added to a runner class and then called

We'll follow this sequence in the illustration. The sample included for download is different.

Writing a Test Case

In this example, we derive our test case from the TestCase base class. TestCase, like the other components of the framework, is a struct. This helps us avoid a lot of public access specifiers.

Test code is added to the single test method. The TestSuite class contains any data that is shared across test cases and it is passed to every test call. The name method is used to provide meaningful output in the event of a test case failure.

One of the benefits of this lightweight system is that all test code may be kept in headers. This avoids some duplication between a separate class declaration and implementation. Because the unit test framework is implemented in a single header, only a single driver module (containing main, for example) is required.

This system also makes it easy to add tests to an existing application. For example, the tests could be called at program startup within a #ifdef DEBUG section. In release mode, no trace of the tests would exist in the application binary. This may not be the case when linking to other unit testing libraries.

Obviously, this is not a long-term solution. It is a good way to get started, though. Developers can start writing tests immediately and the tests can be factored into a separate executable when time permits.

Internals

This section may be skipped. It explains a little bit about how the (very few) moving pieces work.

The class has a name accessor method, which is used for logging errors. The test suite uses the next pointer to chain the test cases into a linked list. The test itself is implemented in the virtual test method.

The test suite has the same structure, except that it contains a list of tests and has different methods to override: setup and teardown.

As stated earlier, the test suite class plays the same role as the test suite and test fixture classes in other frameworks. In such frameworks, suites often play the role of a test grouping construct while the fixture provides a setup/teardown mechanism. Since ShortCUT is such a simple framework, there was no need to create this additional level of complexity. If a development team needs this feature, it may be easily added as a customization.

The Runner

The test runner is the heart of the system. It, too, is very straightforward. The main routine, RunTests calls RunSuite for each suite.

The key points to note here are that, first, the log class can be implemented and set outside of the framework. This makes it easy to display results to another output target, such as a window.

The second point, which can be an annoyance, is that the test suites and test cases are chained together in singly-linked lists. This means that they are traversed and executed in LIFO order. This is the reverse from the order in which they were added.

It would be a simple matter to customize the framework to fix annoyances like this. I chose not to, since the goal was to make the framework as simple as possible.

Customization and Conclusion

The main goal of the framework was to have the absolute simplest system possible, within the design requirements and constraints. Every line of code was scrutinized for its value. In some cases, such as with the TestLog class, a few lines were added because they helped to meet a design requirement. Even though the framework would have been simpler, it would have lost basic flexibility.

The header is about 200 lines of code. A quarter of the code is actually unnecessary. It was included as an example of how to implement custom assert functionality through exceptions and how to implement a couple of helper macros to avoid repetitive code.

The framework is useable in its basic form. It is hoped that it will form the basis of systems that are tailored to the needs of the developers who use them (instead of the other way around). It should provide enough utility to get going quickly, and its basic structure should make it easy to modify, customize, and extend going forward.

Comments and Discussions

CppTest (http://cpptest.sf.net[^]) is IMHO a good alternative. It's a compact package which does not overuse C++ (e.g. templates) and therfore runs on a wide range of compilers. Aside the development is not very active (last version is from 2004) it's stable and provides the most important output handlers.

Roland,
Yes, I merged some of the functionality of the fixture and the suite into the suite in this framework. The fixture concept is often associated with a test case in other frameworks. I chose to merge it the other way towards the suite.

This was a tradeoff. The reasoning was that I didn't want too many levels of classes and associated setup. I also had the constraint that a test had to be structured around a class in order to be kept in a list within the suite. So the fixture function merged upwards in my design. The implications are that when using this framework, if you need to use the fixture setup/teardown feature, then you'll probably create more suites. On the other hand, all of the tests within the suite will be meaningfully related. Many people don't use the fixture feature very often and thus they can just use suites as a plain grouping mechanism.

This all gets back to my original goal of having something that is small and flexible and easy to adapt. I will be experimenting with it myself to see if I can make it even better.

Thanks for the ideas about the private copy ctor and assignment operator. Those three lines of code can help a lot. BTW, the reason for the pointer is to make the (unfortunate) cast more straightforward. Good point about null safety, though.

Since this is a 'Beginner' article, I'd suggest that you add some links in your Introduction section to Web pages explaining what Unit Testing is all about. For example, the Wikipedia entry on Unit Testing[^].

I'm using another lightweight C++ framework that's not often mentioned on-line: Bruce Eckels' book Thinking in C++ 2nd Edition Volume 2[^] has a simple framework that has a slightly different set of design trade-offs from your tool. Not better or worse, I think, just different.

Gordon,
Good idea. I only added a few references related to my search. I'll try to add that when I revise.

I wish I had seen Bruce's book beforehand. He kindly provides it in HTML form on the web, in case others would like to take a look. His test_() macro is very similar to T_ASSERT() in my framework, for example.

A neat system, yes, but "the absolute simplest system possible" can hardly be true.

The system below does much of the same with one header file.

Adding the header file, add one line at global scope, one additional line in a class interface definition and three extra lines in the class implementation, is enough to force the class unit test to be run automatically once and once only for each class constructed. A final additional line, placed anywhere, will summarize test results and exit if any failed.

Of course, this simplicity is achieved by dropping some features. IMHO nothing critical is sacrificed

Your system makes clever use of macros and is indeed a *very* lightweight system. This is interesting material for comparison. Thanks for posting it.

My goal was to provide the simplest vanilla C++ system which met some additional design requirements. This did cause the code to 'bloat' to over 100 lines. Things like adding a TestSuite between the TestRunner and the TestCase did add complexity. I put in that requirement, though, because most unit test frameworks today have setup/teardown mechanisms. Indeed, it is one of the most useful things about SUnit and its derivatives.

My framework also requires a little extra setup in terms of defining a class for each test case. I could have added macros (in fact, I do provide a couple in the header and demo as an example). I avoided them, in general, because, a) many C++ developers dislike macros and, b) people can easily add macros themselves to suit their needs. I also wanted to make it very clear what was going on at the C++ level so that people could customize things more easily.

I am always slightly bemused by test setup/teardown infrastrusture. Most unit tests do not require it. If major setup is required, I think that a regression test at the application level is probably what I am looking at. For the few in between cases, put the setup into a method or class constructor as appropriate and call it from the begining of the test code - what could be simpler or more flexible?

I prefer not to add extra class for every test. My projects are already overflowing with classes that I have to keep straight. If the test-case is a method of the class being tested then keeping it together and up to date is so much easier.

Macros can certainly be over-used, but seem ideal for this purpose. They are certainly difficult to maintain, and I would normally sprinkle them with a lot more comments, to help with maintenance, but I wanted to avoid making this look more complex than it really is, so I avoid mind-numbing explanation of how things were working and just focussed on what they were doing.