novaprova

The name Novaprova comes from the modern Italian word
prova,
one of whose meanings is a test, a trial, an experiment,
and the archaic Italian word nova which like the modern Italian
nuova
means new. Hence, new test.

Disclaimer: I don't speak Italian, but an actual Italian told me this
is a silly but clear reading of the name. Also, it rhymes and it
sounds vaguely like a drug.

Sure it is. All you need to do is to be able to read the debugging
information that the compiler adds to the executable when you build with
the -g option.
The debugger reads that information, so it can't be too hard.

It turns out that almost every modern UNIX-like system uses the highly portable
DWARF standard for debugging
information, so there's little system-dependent code involved.
The only tricky bit is reading that information for your own process,
rather than for another process. Novaprova uses some
runtime linker magic
and
the bfd library
to find the on-disk executable and shared library images for the
program it's linked into (including libraries opened with dlopen),
and some hand-rolled code to scan the debug information.

There are other libraries that claim to do "reflection" for C++, but they
all rely on magic macros or other tricks to explicitly mark classes or
functions. By contrast, Novaprova's reflection can see any class, namespace,
variable or function which the debugger can see, without the explicit
co-operation of the code reflected.

Having real reflection is extremely useful for a test framework. At the
very least it means that test functions can be discovered at runtime
by trawling through all the functions in an executable and matching
function names and signatures.

Novaprova implements function mocking using a technology similar to
debugger breakpoints, but working in the same process rather than in
a child process.

There are several test frameworks that support mocking for C, but
they all rely on the linker choosing to link against the mock
function rather than the mocked one. This has major limitations.

The mocked function must be extern

The mocked function must not in the same object file or
shared library as the calls to it which need to be intercepted
(or special care must be taken to make the mocked function an
ELF weak symbol).

Functions can only be mocked once in a given test executable.

If a function is mocked, it is mocked for all the tests in that
test executable (so the mock function has to be more complex).

The mock functions are installed at build time.

Novaprova's runtime mocking removes all these limitations!

Novaprova's mocks can be attached to any testnode. They are installed just
before the testnode or any of it's descendants is run and uninstalled after
the test finishes. This means you can safely mock functions different
ways in different tests, or mock functions in only some tests, or
mock (almost all) functions which are used by the test framework itself.
You can mock functions that are called from other functions in the same
object file. You can mock static functions, as long as you
know the name of the function or have it's address.

Parameters in Novaprova are static char* variables with a series
of pre-determined string values, which the test code or fixture setup
code can use to control aspects of its behaviour. Novaprova knows which
parameters apply to which test, and will run the test once for each
combination of values of the parameters, setting up the parameters
before each run. The result from each run is reported separately.

A test executable which uses Novaprova can be built like any other
C executable. Tests are written in C, and no special build-time
work is necessary (the magic all happens at runtime).

Even though Novaprova is implemented using C++ internally, it has a
full C API so that tests can be implemented without a C++ compiler,
just a C compiler. This is important when you have older code whose
headers may or may not compile cleanly under C++.