C/C++

Testing Complex C++ Systems

By Gigi Sayfan, January 29, 2013

Some systems are so complex that they require non-traditional approaches to thoroughly test large code bases.

Solutions

My favorite technique for fast C++ tests is to move from the traditional monolith with lots of static libraries to a component-based approach, where the application is composed of DLLs or shared libraries. (To this end, my series of articles on C++ plugins explains how to do this using a plugin framework.) Then the tests themselves are DLLs, which can be built and executed as needed by the test framework. This requires a lot of foundational work and is suitable either to green-field projects or those where upper-management supports re-engineering the system.

Webcasts

As to the weak type system, if you are able to build (or refactor) the system, you are on solid ground here. All the techniques I demonstrated in the first article are applicable to to C++, too. Interfaces in C++ are just classes/structs with only pure virtual functions. Be careful about the data types you use in the signature of your interface methods and be mindful about the public interfaces you expose from your components.

Testing Non-Public Code

Testing non-public code in C++ is a controversial topic. One camp says you should never do it and only test the public interface. I agree with this sentiment and try to accommodate it by creating small components with clear interfaces that are easy to test from the outside. However, every now and then, you will have a component with complex internal machinery that is not easy to test via the external interface. One way to get to the private parts of classes is via the dreaded "friend" keyword: a C++ class that may grant access to its private parts to other classes. Another approach is to add public introspection methods (perhaps protected by a TEST symbol that requires a special test build and are not available in production).

Be careful when using these techniques. Friendliness can be dangerous, and requires your production code to know of the code that tests it. If you decide to add another test, you need to modify your production code. Also, if developers see "friend" is used, they might decide to use it to provide access to other production classes instead of applying best practices. It can become a slippery slope. Providing TEST-only methods to introspect internal state is not ideal either. Again, some "clever" developer may decide to define the TEST symbol in his production build and call your TEST-only methods in production. Also, the TEST-only code may have unintended side effects. Since you don't test your code without it, you don't know how your code will behave in production.

One other really nefarious technique is to use macros to redefine the "private" and "protected" keywords to "public." This is very hackish and requires a lot of attention, but at least you can do it from within your test code. Here, class A is defined in a header file called "A.h":

class A
{
public:
A() { x = 5; }
private:
int x;
};

The following test code is able to access and even modify its private x variable:

This technique is, of course, super fragile and should be used with caution. It can break in several ways, but the good news is that the compiler will tell you about it. One way it can break is if the private state uses the default "privateness" of C++ classes:

class A
{
int x;
public:
A() { x = 5; }
};

If the private state is defined at the top of the class without any access modifier, it is private by default, but there is no "private" statement in the class that you can redefine to "public." If you have access to the code, you can just add the "private" keyword. Another way it can break is if another class #includes "A.h" and it is protected with an #include guard. In this case, the first definition of A will have x as private and your trickery will not work.

You can redefine "private" to "public" as the first statement in your test's main function, but if you link with prebuilt static libraries or DLLs, you'll have to ensure they are built with redefined "private," too. As I said, it's a very brittle technique.

Real-Life Scenario: Testing Rockmelt

Rockmelt is a social browser built on top of Chromium  the open source browser behind Google Chrome  which, itself, is built on top of WebKit and other open source projects. WebKit has plugins, Chromium has extensions, and Rockmelt has "Extended" extensions, which enable extensions to access Rockmelt-specific functionality. Rockmelt also has a big back-end component that the browser communicates with. If you think it's a bear to manage, you're right. Chromium releases a new version every six weeks and Rockmelt has to follow closely. If Rockmelt falls behind, then the browser is vulnerable to public security issues that were fixed on Chromium, and extensions that rely on Chrome might stop working. A significant part of each iteration is dedicated to upgrading Rockmelt to the next version of Chromium. Rockmelt is not just Chromium + a few extensions. It is a highly customized version that reaches deep into Chromium (all back-end communication, synch framework, tons of custom UI, Rockmelt JavaScript bindings, etc). It is developed in C++ and supports Windows and Mac OS X.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!