What is meant by Structural (Code) Coverage?

Put simply structural, or code, coverage is the amount of code that is covered in execution by a single test or collection of tests.

For a procedural language like C, you can identify a function of interest, run some test cases on this function, and then measure what proportion (expressed as a percentage) of the code has been executed. The general rule is that the higher the coverage achieved, then the higher the confidence that it has been thoroughly tested.

Measurement of structural coverage of code is an objective means of assessing the thoroughness of testing. There are various industry-standard metrics available for measuring structural coverage, these can be gathered easily with support from software tools. Such metrics do not constitute testing techniques, but a measure of the effectiveness of testing techniques.

A coverage metric is expressed in terms of a ratio of the code construct items executed or evaluated at least once, to the total number of code construct items. This is usually expressed as a percentage.

There is significant overlap between the benefits of many of the structural code coverage metrics.

Structural code coverage is a measure of the completeness of software testing showing which areas of the source code are exercised in the application during the test. This provides a convenient way to ensure that software is not released with untested code.

The list below identifies reasons why some code has been found to be untested using structural code coverage and the resulting actions which may be taken.

The fundamental strategic question of how much testing you should do is generally driven by available resources, both time and budget. If you are not required to measure tests against a specific set of structural code coverage metrics by a software safety standard, then the choice of which metrics and which thresholds to set as acceptable can be determined by your own software quality policy. For more information on the advantages and disadvantages of different code coverage metrics see the QA Systems white paper “Which Code Coverage Metrics to Use”.

For all the main software safety standards the required code coverage metrics (depending on integrity level) are Entry-point, Function Call, Statement, Decision and MC/DC coverage. These are explained in more detail below.

Entry-Point Coverage

Entry-Point coverage measures the proportion of functions in the source code that have been executed at least once. It is the easiest metric to achieve 100% coverage in tests.

Call-Return Coverage

Call-Return coverage measures the proportion of function or method calls in the source code made and completed at least once. It is the most commonly used metric to measure integration level testing.

Decision Coverage

Decision Coverage measures the proportion of decision outcomes in the source code which have been evaluated at least once. It can sometimes be referred to by these alternate names: C2, Branch Coverage, TER2, TER-B coverage. Decisions include constructs such as ‘if… else…’, ‘switch… case…’ and loops such as ‘while’ and ‘for’. Decision coverage contains Statement coverage but ignores the complexities of conditions within decisions.

MC/DC — Modified Condition / Decision Coverage

Modified Condition Decision Coverage measures the proportion of operand conditions which could independently affect the true/false outcome of the decision expression that have been effective in doing so at least once. It can sometimes be referred to as a combination of Decision coverage and Boolean Operand Effectiveness coverage. MC/DC coverage demonstrates that every sub-condition can affect the outcome of the decision, independent of the other sub-condition values.

There are two methods for measuring MC/DC coverage: Unique Cause and Masking. The latter was created by Boeing to accommodate the short-circuiting evaluation of true/false expressions in C/C++.

MC/DC is the hardest metric to achieve 100% coverage in tests requiring the most test cases.

Do you want to to know more?

If you found this introductory article useful, QA-Systems have a detailed paper available for download for free here. This expands on the principles touched upon in this short article.