In the two previous posts, we started the development of a dynamic library on Linux: the first one saw us building the library and managing version numbers using symbolic links, in the second one we traced library calls and did step-by-step debugging. Now we are interested in checking the coverage of the library.

Code coverage is a measure indicating the percentage of lines of code that were actually covered during a program execution. We can gradually expand the set of tests to obtain 100% coverage (and ensure that the code has been fully verified).

The best known tool under Linux for code coverage is gcov, which instruments the source code and provide us detailed statistics after execution. It is easy to use it to verify coverage of a source file compiled into an executable. We will use it for library code which requires some more attention.

Compilation

Let’s start with the same files and directories (grouped in this archive) as in the previous articles. A directory named “factorial” contains four subdirectories: src, include and lib respectively where the source files, header files and compiled files of the library are. The fourth sub-directory “test” contains the source and executable files of a program using our library libfact.

NB: The library was already compiled in the above example, but we will rebuild it.

At first we will compile the library code, as in the first article, but adding the --coverage option of gcc. This option has two different roles:

During compilation, it has the same meaning as -fprofile-arcs and -ftest-coverage options (which were used with the previous versions of gcc): the first one added instrumentation data to the executable code (counters), the second one created a table of correspondence between instructions blocks and lines of source code (in a file named after the source file with the extension .gcno)

During linking, it is equivalent to -lgcov (which was added automatically by -ftest-coverage) that incorporates the necessary entry points for the subsequent use of gcov.

We see that with the --coverage option, the compilation generated, in addition to the fact.o object file, a fact.gcno file, containing the relationships between the blocks of code and the line numbers. We continue.

We have rebuilt the libfact.so.2.0 library. Symbolic links are used to manage the major and minor version numbers, as we saw in the first article. Now compile an executable file, without --coverage option (or use the executable file of previous articles).

Execution

Program execution takes place quite normally (although in practice it is slightly slower). We must, of course, set the environment variable LD_LIBRARY_PATH to specify where the dynamic linker will find the library needed to run the application.

gcov tells us that we have only performed 87.5% of the eight lines of code in our function. What happened? We see that gcov also created a file named “fact.c.gcov” in which he puts a copy of our source code, numbering the lines, adding a header and a column of statistics at the beggining of the line.

The header describes the files involved and the number of executions (only one here). The left column shows the number of passes on each line. Lines containing a dash “-” do not match any compiled code. We see that lines 3, 5, 6 and 12 were scanned three times (one invocation for each argument on the command line), and that lines 9, 10 and 11 were executed 12 times (iterations to calculate factorial).

Unlike spreadsheet programs, this symbol does not mean that the number is too large to fit in the column, but that the line (which corresponds to compiled code) was never executed. Two advantages with this notation:

it attracts the eye better than a single “0” would do,

it allows us to do an automated search of unexecuted lines using grep.

Now gcov tells us that all the lines of our program have been covered by our test set. This reduces the probability of remaining bug.

Error handling

A major difficulty to achieve 100% code coverage during software testing is to validate the behavior in case of system error.

Take a look at a well-known system call: malloc(). We asked him to allocate a memory area of ​​a certain size (given in bytes) and he returns a pointer. All documentation tell you that in case of lack of memory, malloc() returns a NULL pointer. (Although this case is particularly difficult to produce with Linux, we will discuss this in a future article). Also, the conscientious programmer will write something like.

Unfortunately the lines in between braces are difficult to test because we can not “force” malloc() to fail. The failure circumstances are based on too many parameters external to the application to be reproducible.

In the specific case of malloc(), the Glibc library provides us entry points that can be used to replace the function – see malloc_hook(3) – but it is not possible for other system calls.

However there are several solutions. One of them, I have used several times, is to use a software layer that replicates the minimum system calls we need and simulates a failure if certain criteria are met. For example the following routine reproduces malloc() but fails after a number of invocations contained in the environment variable MALLOC_FAIL.

In this way, depending on the presence or absence of the NDEBUG constant, which traditionally represents for the C library the production version of the code, our routine will be compiled as usual malloc() or with our management of the environment variable.

Of course, this principle of forcing system call failures under the control of an environment variable – or other parameters (global variable, file, shared memory area, etc.) – can be applied equally to a library code when you need 100% code coverage over the entire set of tests for an application.

Conclusion

We observed in this small series of articles, how to create, debug and test a dynamic library. I encourage you to do your own tests, referring to the documentation of gcc, gdb, gcov, but also other complementary tools such as gprof, ldconfig, valgrind, etc.