Posts From November 2013

I was recently asked in an email if I knew of any tools that would translate the XML results file from the JetBrains ReSharper command line tool inspectcode into a human readable format, such as HTML. For your viewing pleasure, here was my response:

There aren't any tools out there yet to convert the XML to HTML, but the XML format is fairly simple, so I suspect it wouldn't take much to write your own.

The file is broken into two sections <IssueTypes> and <Issues>

Under <IssueTypes>, there's a collection of <IssueType> elements, which list all of the violation types that were discovered in your code. Each one has the following possible attributes:

Id

This is the unique identifier for the rule

Category

This is a general grouping for the rule types

SubCategory (optional)

some of the groupings are further split

Description

This is a general description of what the rule is checking

Severity

One of these values: ERROR, WARNING, SUGGESTION, HINT, DO_NOT_SHOW

WikiUrl (optional)

A link to a jetbrains webpage that has additional details about the rule

In the <Issues> section are the specific instances of rule violations found in your code. Under <Issues>, you will find a collection of <Project> elements, each with a Name attribute which will match your Visual Studio project name. Under each <Project> attribute will be a collection of <Issue> elements, each with the following attributes:

TypeId

A reference to the rule in the collection, where the TypeId here is a match for the Id in the element

File

The file path that contains the violation. This path is relative to the Visual Studio Solution's folder

Line

The line number in the file where the issue occurred

Message

A message about why this line of code violated the rule. The message is case-specific and often include the variable name or other context information specific to this line of code.

Offset

I'm 100% sure what this actually represents, but from what I can tell, it's the character range (offset from start of file) of the specific text in the file that violates the rule. In Visual Studio, this would be the text that is highlighted/underlined, etc. So a value of "1684-1722" would be the 1684th character in the file through the 1722nd character.

Before I open the gates and push a formal release to the public, I'd like to get some beta testers to use the plugin and work out any rough edges. If you're using SonarQube for your .Net-based projects, I'd love to have you as an early adopter and get your feedback.

Overview

The basic idea is this:
The good folks at JetBrains have released a commandline-based tool (called "InspectCode") allowing users to run ReSharper code analysis outside of Visual Studio and generate an XML-based file listing the various issues it detected in your codebase. Oh, and it's free!

I felt this lined up very well with SonarQube, and wrote a SonarQube plugin to (optionally) execute the ReSharper analysis and consume the results, publishing them into SonarQube as Issues, allowing users to take advantage of the management, planning, reporting, etc, features of SonarQube to manage their code quality and technical debt.

The official documentation for installing this plugin can be found on the SonarQube Wiki page for this plugin, which I'm still building out, and will change over time to reflect the current state of things. For now, here's the process:

Download resharper EAP and run a local analysis
First, make sure the InspectCode tool is working for you. Download the ReSharper Command Line Tools (they come as a .zip file) and extract them into the folder of your choosing. For the purpose of this posting, I'm assuming you used C:\jetbrains-commandline-tools\.

Open a command prompt and run inspectcode.exe /help. You should see a usage statements printed to the console.

Now change into the folder with your VS Solution file and run the command c:\jetbrains-commandline-tools\inspectcode.exe /output=resharper-results.xml YourSolutionFile.sln

Warning: This may take a very long time to run, possibly a couple hours for very large projects. I would suggest starting with a small project and maybe grabbing some lunch.

One this completes, you should have a resharper-results.xml file in your solution folder, which will look a little like this:

Download the sonar-dotnet-resharper-plugin from the build server and place it in your SONARQUBE_SERVER_HOME/extensions/plugins folder.

Restart your SonarQube service

Login to the SonarQube server with an admin account and head over to the Quality Profiles page and select the profile used by your .Net project (this may be the "Sonar way" profile, or one you've created on your own).

In the search fields, set the Repository to ReSharper and the Activation to Any and click "Search". You should be presented with a list of approx 650 rules supported by ReSharper's tool. In the upper right corner of the results list is a Bulk Change option. Choose Activate all. This will (immediately) activate all of the ReSharper rules for this profile.

Execute sonar-runner and get yourself a cup of coffee. For really big projects this could take half and hour or more.

Bask in the glory of your new ReSharper-based Issues in the SonarQube web UI.

A Note About Rule Severity

ReSharper users a four-tier severity (plus a "Do Not Show" option), while SonarQube uses a five-tier system. I have mapped the ReSharper values like this:

ReSharper

SonarQube

Error

Blocker

Warning

Critical

-

Major

Suggestion

Minor

Hint

Info

Do Not Show

Info

While some of the ReSharper rules are marked as Do Not Show by default, the plugin maps them to Info so that you can modify their visibility via the Quality Profile. If you want them to not show, disable the rules in the Quality Profile. You can, of course, change any of these levels for any rule using the Quality Profile, and I fully expect most people will want to downgrade a lot of the Critical rules to Major and a lot of Info to be turned off.

Reuse Report vs Run During Analysis

The ReSharper analysis can be very time consuming for large solutions with many projects, sometime taking multiple hours to run if the cache has to be rebuilt, so I would suggest users favor the reuseReport mode when possible. This reduces the number of times the analysis is executed, since running it during the sonar-runner will analyze each project separately, adding significant analysis time, even with heavy caching in the ReSharper tool.

To use the reuseReport option you'll need to use the inspectcode.exe utility to analyze your solution before executing sonar-runner. There are several command line options for the tool, but at minimum, you’ll need this:

inspectcode.exe -o=resharper-results.xml YourSolutionName.sln

Where -o=resharper-results.xml will write the results into the file name you provide. Note: by default (without the -o argument), inspectcode will write the file to the temp directory. You’ll need this file to provide to the plugin.

Analysis times

The amount of time required to run the inspectcode analyzer and import the results into SonarQube can vary based on several factors. Here is some representative data to give you some guidance on how long it will take, based on two actual projects I'm analyzing.

Small Project

Larger Project

Lines of Code

27,515

593,979

VS Projects in Solution

10

39

Standalone InspectCode

4min

1hr 24min

Sonar-Runner (ReuseReport)

2min

35min

Sonar-Runner (embedded InspectCode)

11min

2hr 58min

All cases had an existing ReSharper cache. Excludes unsupported project types, such as WiX (installer) projects. Includes only the .NET ReSharper plugin and the base C# plugins (ie: does not include FxCop, Gallio/Unit Tests, etc). Timings are based on running within a TeamCity CI build on a decently powerful server. Obviously, your times will vary and these represent only a guidepost for how much variation can happen between runs.

Known Issues / Limitations

The ReSharper command line tools are fairly new and still have a few kinks to work out. I would suggest always using the newest possible version, including new EAP builds of 8.1 while it is still in development.

Inconsistent results from inspectcode

From one run to the next, you may get a different list of rule violations from ReSharper, even without code changes. This means you will have violations come and go in SonarQube without the underlying issue actually being addressed. In my tests, for a large project like the Windows Azure .net library, I saw a 4% difference (~430 rules delta on ~12,000 total)

Data quality issues in available rules

With the 8.1 EAP builds, JetBrains has added a commandline switch to inspectcode to dump the list of possible issuetypes that could be reported. There are a couple of data issues in that output, which I have worked around in the plugin:

"Sonar.UnknownIssueType" Violation

SonarQube requires that the rule database be populated up front, at server start. ReSharper provides a list of possible violation types for their tool via the /dumpIssuesTypes argument, and I've populated the SonarQube rule repository using these values. However, as new versions come out, or you add your own ReSharper plugins to generate new IssueTypes, it’s possible that you will have violations in your code that the plugin did not know about when the server started, so cannot import into SonarQube appropriately. When the plugin comes across one of these, it will write a log statement (to the runner’s log) and generate a "Sonar.UnknownIssueType" violation with instructions on how to add the rule information to the 'ReSharper custom rules' property in the Settings UI. After adding that setting and restarting SonarQube, future analysis runs will support those IssueTypes.

Other File Types

While the ReSharper tools support the full .Net stack, including .xaml and other file types, The sonar-dotnet API does not currently support more than one language (C# or VB.net) for a given project. For .xaml and other files, the plugin attempts to report the violation against the file, but they may end up attached to the project instead. In either case, the source code for the .xaml or other unsupported file type will not be imported into SonarQube.

Future plans and release timeline

There are a handful of things I want to see before I release version 1.0, which will go out in the Update Center for general public consumption:

ReSharper 8.1 released. I feel like the 8.0 release of the command line tools just isn't viable for use with SonarQube, primarily due to the inconsistent results from one run to the next. This also ensures any breaking changes they make before 8.1 ships are caught and addressed in the plugin. JetBrains has not publicly stated when they thing 8.1 will be released.

A handful of early adopters are using the plugin and providing feedback (and are happy with it).

I've addressed any high-value tickets in JIRA. I'm using the NETRESHARPER-1.0 release to track things targeted for the public release.

Other features I'm hoping to include, but are not guaranteed:

additional configuration settings to modify the inspectcode arguments when running directly. For example: ability to set the cache folder path, disable solution-wide analysis, use of a R# settings file, etc.

utilize and document ReSharper plugins in the inspectcode analysis. This is a core feature we're looking to utilize at work to make the dev-time experience match the CI server analysis for company coding standards enforcement. This may "just work", but is dependent on JetBrains releasing the 8.1 SDK nuget packages, which will be "later this autumn".

Disclaimers

I am not associated with JetBrains in any way, aside from being a big fan of many of their products. I have, however, received free licenses to JetBrains products (ReSharper and dotPeek) over the years via raffles over the years at the Chicago Alt.Net Users Group, which is sponsored by JetBrains.