I'm a Principal at ThoughtWorks, classical composer, and physics amateur.

Tuesday, May 01, 2018

JaCoCo, Gradle, and exclusions

The setup

My team is working on a Java server, as part of a larger project
project, using Gradle to build and JaCoCo
to measure testing code coverage. The build fails if coverage
drops below fixed limits (branch, instruction, and
line)—"verification" in JaCoCo-speak.

We follow the strategy of The
Ratchet: as dev pairs push commits into the project, code
coverage may not drop without group agreement, and if coverage
rises, the verification limits rise to match. This ensures we
have ever-rising coverage, and avoid new code which lacks
adequate testing.

The problem

At a work project, we're struggling to get JaCoCo to ignore some
new, configuration-only Java classes. These classes have no
"real" implementation code to test, are used to setup
communication with an external resource, yet are high
line-count (static configuration via code). So they drag down
our code coverage limits, and there is no effective way to
unit test them sensibly. (They are best tested as system
tests within our CI pipeline using live programs and remote
resources.)

JaCoCo has what seems at first blush a sensible way to exclude
these configuration classes from testing:

Something to consider: using wildcards
(hm.binkley.labs.saml.*) may take additional work.

Why?

Why does this work, and the "obvious" way does not?

JaCoCo has more than one notion of scoping. The clearest one is the
counters: branches, classes, instructions, lines,
and methods.

Not as well documented is the
scope of checks: bundles, classes, methods, packages, and
source files. These are not mix-and-match. For
example, exclusions apply to classes. Lyudmil Latinov has the
best hints I've found on how this works.