Filtering with dotCover

dotCover allows us to run coverage analysis on our code. However, there are times when we do not want to perform an analysis on certain areas. This could be our test assemblies, certain third-party assemblies or even specific parts of our own project. Since the analysis has an impact on the overall statistics and potentially can take longer, it is often interesting for us to filter certain assemblies or classes out.

The figure below shows the coverage report of the default MVC project (yes, yet another MVC example, but it ships in the box and probably the ONLY template with VS that comes with tests). In this case we’re using xUnit for tests, which is mostly to demonstrate that dotCover works with other frameworks, not only MSTest (personally I’ve moved on to MSpec, which dotCover also supports).

If we focus on the Coverage results we can see that there are some areas in grey, which indicate that the corresponding PDB files were missing so coverage could not take place. We can also see other assemblies that have been included in the results yet might not be of interest, such as the test assemblies.

In order to filter these out, we can use the Coverage Filters, which can be accessed via the dotCover menu in the IDE

By default, everything is set to be covered, as shown by the Everything entry in the Allow Filters tab (which most likely will be renamed to Included). Examining this entry (clicking on Edit) we can see that it is composed of three values:

Module Mask indicates the project name

Class Mask indicates the class name

Function Mask indicates the method name

(All entries support wildcards as displayed by the * that appears)

We can now uses these patterns to exclude projects, assemblies, namespaces, classes and methods from out tests.

Excluding entire project

Click on Deny Filters tab (most likely will be renamed to Excluded) and click on Add, entering the following information:

Clicking OK and re-running the tests, we now see that the entire test assembly has been excluded

Excluding an entire Namespace

We can exclude entire namespaces by setting it in the Class Mask as shown below:

this will exclude all the classes inside the MvcApplication16.Models namespace. If we just wanted a specific class, we add it after Models instead of *.

Excluding a method

We can exclude a method, similar to how we’ve done it with the previous cases:

Adding a series of filters for test and auxiliary assemblies using the previous steps, we can end up with a cleaner and more accurate coverage report:

produced by adding the following filters:

Notice that by using the *.Tests (or alternatively as I call it *.Specifications), we can automatically exclude all our test projects from code coverage.

Beta Disclosure

Note 1: Currently these settings are Global, that is, they apply to all projects. Most likely we’ll be changing it so that they are Solution and Global scoped. This way, common libraries can be excluded for all solutions, and specific projects and/or namespaces can be solution based.

Note 2: We’re also looking at adding functionality to make it easier to define these exclusions (a la Right-Click on Folder, add to Excluded list)

@Brian,
Not sure about that one but I’ll check (or someone else will follow-up). However, can I ask, why not use filters? It’s less intrusive and does not pollute your code with framework specific attributes.

I would really like to be able to use the [CoverageExclude] attribute. Without this, I probably wouldn’t recommend it for my team. We have a huge amount of code and use this attribute to signify to nCover to ignore the type.

I’m not a great fan of attributes, but I find this particular attributes makes the intent of the code clearer, e.g. ‘this component physically touches external dependencies and can’t be tested’ (whether right or wrong!)

Hadi, thanks. I pasted the wrong snippet of code. The above code IS covered by my xUnit tests, but for some reason, when I run the whole test suite at once, it says it’s not covered.

I think this is because I do quasi-bdd style tests, something like:
public class FooSpecs
{
public class WhenFooIsConfigured
{
public class And_someone_calls_it_who_IS_permissioned
{
[Fact]
public void It_should_do_something( )
{}
}

The R# test runner sometimes doesn’t run the test when I right click on FooSpecs, sometimes it does. Also, if I dotCover the tests manually and then dotCover the whole test suite, it says that previously covered code is no longer covered.

Is it possible to exclude all files ending with one given pattern? For instance I would like to exclude from the coverage analysis all files in the solution matching the pattern *Specs.cs (i.e ending with Specs).

The reason for that question is that our tests are defined in the project where the object being tested resides so that we do not have to duplicate the folder structure in a separate test project.

@Haidi,
I had tried *.Specs, *.Specs.cs etc.. already but unfortunately this does not work.
Filtering each folder would theoretically do the trick, but would be way to much work.
Is this a filtering feature that you guys might consider implementing?

I have been using dotCover for about a month now.
You mention that this is a beta product and that you aim to make it easier to manage filters in future releases so here are some of my suggestions and comments based on what I have played around with so far:
I like the right-click folder, project, etc to add/remove from coverage filters.
It’d be nice to have this in both the solution explorer and the coverage results window, this way I can manage down to the function level.
Would like to see a folder/file/path name filter option as well. Using WCF and other MS tools that auto generate code with specific filenames can be filtered out. For example one could exclude all Reference.cs below the root solution folder.
for reporting I think it could be useful to keep a histogram counting the number of times a method is called during the analysis. This way if I have a few methods that are not covered well, but one that is called more frequently I can concentrate effort there as it is likely a more significant point of potential failure.

I’m using dotCover 2.0 and still don’t get why setting up filters/exclusions is so hard.

The new UI where I can set up filters IS great. Fantastic even – as it’ll let me use wildcards and so on. So that’s a GREAT win.

BUT, I still see a HUGE disconnect in the UI.

Right now in the Coverage Browser, I can go in… right click on entire assemblies, namespaces, classes, methods, and so on – it’s INSANELY granular and VERY easy to use.

Then I can save said ‘coverage’ to a .dcvr file.

What I’d LOVE to do?

Be able to LOAD that back in and have THAT be what is used for setting all of my filters. Or, maybe some other file/etc.

But it seems to me that I should be able to just save a ‘MySolution.dcvr’ (or .whatever) and have THAT be the more-or-less ‘defacto’ list of coverage that I care about when I re-run coverage. Ideally, there’d be some sort of convention that would try to re-load that per solution… but even WITHOUT that option if I could just use the dotCover > Open Coverage Snapshot and have THAT remember all the crap I excluded before AND have that be what shows up in the Unit Tests window/Coverage tab?

THEN this would be IDEAL for me.

Because, as it is, I still find myself scratching my head each time I open this up, exclude a few classes/assemblies I don’t care about and then see the option to SAVE and then LOAD these ‘snapshots’ without ANY of the state (i.e. exclusions/filters) persisted.

I second what Michael Campbell said. The disconnect between the test runner’s coverage tab’s filtering abilities and the dotCover 2.0 Edit Coverage Filters screen is just weird. I’ll add a feature request, but I’m sure there’s some reason you guys aren’t doing this because it seems really, really obvious that it’s a huge win for your users if both UIs can generate permanent filters.