Posts From November 2011

NOTE: This post was last updated for SonarQube v4 in Dec 2013. The support for .NET has changed quite a bit since then,
so this guide is probably not going to be very useful anymore. Sorry.

In an effort to better understand some of the problematic areas of the C# codebase I work on, I recently setup an instance of the
SonarQube code analysis platform. SonarQube is originally written for Java analysis and later added C#
support. This posting walks you through my experience attempting to setup, configure and run the analysis.

Note: SonarQube changed it's name from "Sonar" in mid-2013, so older references to this posting may use the old name.

I periodically update this post to reflect changes with newer versions of the tools. Most recent update was 12/18/2013
based on a fresh install of SonarQube v4.0.

I've also written a SonarQube plugin to use ReSharper as a source for quality metrics. Once you have your SonarQube instance
up and running for your .NET project, see my post
SonarQube .Net ReSharper Beta Release for details on importing ReSharper results
into SonarQube.

SonarQube Overview:

So why did I even do this? Once up and running, SonarQube provides some useful metrics for pointing out hotspots in your code
that may be making it more difficult to maintain and extend your functionality. Through the web interface, you can drill-down on
any of the metrics to the module, class, and method level, including full source code. Some of the metrics provided for each C#
project include (screenshots are from the "nemo" demo site mentioned below, my own project, or other sources):

In addition, it will track changes over time, so you can see where issues are increasing/decreasing in your code.

SonarQube is open sourced under the LGPL and free to use, however some of the plugins used to perform the analysis are only
commercially available, and in some cases come with steep licensing fees. For this blog, I focus on only freely available (ie: no
fees) aspects of the product. Each of the tools I used are also freely available (FxCop, StyleCop, Gendarme, Gallio, OpenCover,
MySQL -- all have licenses that allow no-fee usage for most people).

There is a demo site provided by one of the commercial plugin providers to demo the system
(including their not-free SQALE plugin) which shows analysis for several open-source Java projects and can give you a feel for the UI
and the data that can be provided. Beware -- this includes data from some of the commercial plugins, so don't expect to see
everything on that site after following this posting.

You can also reference the [official installation notes])http://docs.codehaus.org/display/SONAR/Setup+and+Upgrade).

General SonarQube Technical Architecture:

Generally, there is a "runner" that consumes the source code and analyzes it via plugins. This information is published to the
SonarQube database directly. Before each run, the runner makes a call to the server to download configuration settings stored there,
and mashes those with configuration settings stored in a local config file, as well as a project-specific config file.

The SonarQube (web) server pulls the results from the database and provides a UI for reviewing and managing them, as well as a UI for
managing configuration settings, user authentication, etc. The server also has it's own config file where you define database
settings, user authentication settings, etc.

Step 1: Prerequisites and Assumptions:

This posting assumes you have a working Debug build of your codebase that compiles with no errors and generates .pdb files. You need
access to the source code and the output, as they exists at the completion of your build. In my case, I have a CI server
(TeamCity for my playground site -- free for small/mid-scale configurations, and
Jenkins for our production BI build servers) running and just added another build step at the end to kick-off the SonarQube analysis
from the command line.

For many of the plugins (such as FxCop and Gendarme), you'll need to install the .NET Reference Assemblies, which are part of the
Win7 SDK for versions of .NET 1.0 through 4.0 and the
Win8 SDK for .NET version 4.5. Note that the
FxCop installer is included (exclusively) with the Win SDK. Also note that a full Win7 SDK install is several Gigs in size and will
take some time to download, so plan accordingly. You only need to install the Reference Assemblies (under the .NET Development header
in the installer) which ends up being about 151MB to download and 505MB to install if you use the web-based installer I linked to
above.

If you have any Silverlight components in your build, you will need to install the
Silverlight SDK for the version of Silverlight you are
targeting.

Install the Java JDK
Yes, the JDK. It's big and ugly, I know. You're a .Net dev, I know. Do it anyway. Set theJAVA_HOMEenvironment variable to point to the JDK root folder. This should be the “jre” folder – for
example, on my machine the value is set like this: JAVA_HOME=C:\\Program Files\\Java\\jre7 Note: You don’t
need to install the Source Code component from the installer, which will save you about 500MB of disk space.

If you’re using Gallio for running your unit tests, you’ll also need to make sure .NET2.0 is installed. For newer versions of
Windows (Win8 and Win Server 2012), this requires going into “Turn on Windows Features” settings and adding the “.NET Framework
3.5 Features” feature (note: in WinServer 2012, this is part of the Application Server role).

Step 2: The SonarQube Server

This will give you a .zip file. Decompress it into the location you want; I used C:/sonar-server

For the initial configuration/setup, I'd suggest running it at the command line until you know you have it fully configured.
To do that, open a command window and run bin\\windows-x86-64\\StartSonar.bat (or -x86-32 for 32-bit
machines) This will run in your current window (no real output, but the command blocks) until you hit Cntl-Z or Cntl-Break. You'll
see it pulling down additional packages as needed, such as JRuby, but eventually you'll see something like this, which means it's
working:

Once you get things working, I'd suggest installing it as a Windows service. To do this, first register Sonar as a service
using the provided script: bin\\windows-x86-64\\InstallNTService.bat (There's an UnInstallNTService.bat
script too). Then, start the service from the Windows Services control panel, or using the provided StartNTService.bat
script. Personally, I would suggest just configuring it to be Automatic so it will survive reboots, etc.

With my SonarQube install on Windows Server, running the service failed initially, so I used the logs/sonar.log file to troubleshoot
and found this error:

Initially, I just took the lazy route and configured the service to run under my user account instead of a system account, so didn't
really "fix" that issue.
Update: In a later installs, I had to create the “Temp” folder in the systemprofile\AppData\Local folder and
provide security rights to the LOCAL SERVICE user.

The server has a sonar.properties config file that drives some of it's functionality, including which database to use.
You can find it in the \conf folder. The only change I made to this file was the port used for web connections (default is
9000 – I changed to 80), and the database (see Step 3) -- but if you want to use the built-in Derby database, no changes are needed.

Step 2a: Verify and Change Password

At this point, you should be able to access the SonarQube server at http://localhost:9000 (or whatever port you set in the
sonar.properties file). The default username/password is admin/admin. I would highly recommend changing the
admin password at this time. Passwords are changed via the user profile page.

Step 3: The Database

SonarQube server comes with a build-in Derby database. While it is quick to install and easy (and pre-configured), I would
suggest moving away from it almost immediately. I had MAJOR performance problems using it. For instance,
for my solution of about 400K lines of code (50+ Visual Studio projects), when using the Derby database, it took about 2 hours to
perform the analysis, 5 hours if the runner was on a different host. During that time, the server CPU would be pegged at 100% for
5 - 15 minutes per project as the runner reported it's results. Once I moved to a MySQL database, it takes about 20 minutes
total.

So, I would suggest downloading and installing the free MySQL Community Edition
from Oracle. At the time of this (revised) writing, it was at version 5.6.13.

Once installed, you'll need to create the SonarQube database instance. This is pretty straightforward, but SonarQube provides
a script to make it even that much easier. For v3.7, from the SonarQube installation folder, the script is in
extras/database/mysql/create_database.sql. It's no longer included in the v4.0 release, but you can find it
on the SonarQube GitHub repository.
The command to execute the query is at the top of the file. This will generate the SonarQube database instance, as well as grant
access from localhost and remotely. The SonarQube schema is created the first time you run the server.

You'll need to configure the SonarQube server to use this db. This is as easy as changing a couple of lines in the
sonar.properties config file. Chances are, the lines you want are already in the file (commented out), as samples are
there for MySQL, PostgreSQL, Oracle and MS SqlServer. (Note: MS SqlServer is not officially supported, although people on the web
have posted ways to get it working. I did not venture down this path).

Update: While getting SonarQube up and running in our production environment, the person doing the work was able to
get up and running with MS SqlServer and provided me with this feedback. Note, he was using v.2.13.1, which is newer than what I was
using at the time:

I should note that there isn't a quick-and-easy way to migrate from the Derby db to another, so expect any data collected in Derby
will be lost when you move. I found this page
where someone found a way, if you really must try.

Note: at one point, I was getting a com.mysql.jdbc.PacketTooBigException error, so I followed
the suggestions here and ran the sql command:
SET GLOBAL max_allowed_packet = 1024*1024*14;

Step 3a: Verify and Change password, again

At this point, the SonarQube server should be up and running again, albeit with no data. I would suggest verifying you can reach
the server at http://localhost:9000 (or whatever port you used in the sonar.properties file). The user accounts will have reset
with the database change, so again I would highly recommend changing the admin password at this time!

Step 4: The Runner

SonarQube was build for Java, so the docs almost all assume you're using Maven. You may be able to get this working with nAnt,
but there's an easier route. A small, java-based runner
has been created that can be kicked-off from the command line. This is what I used from within my TeamCity and Jenkins builds.
There’s also a Jenkins plugin that looks promising,
but I haven’t used it.

To use this runner, you will need to do a few things:

Download the runner. At this (revised) writing, the version was 2.3. This will come as a Zip file. Decompress it in an appropriate location. For me, I put it in C:/sonar-runner/

Test that it works. Open a command line to the sonar-runner/bin folder and run sonar-runner -h. This will either show you the usage statement (if things are working fine), or blow up. If it blows up, check you'reJAVA_HOME is set correctly to point at the jre folder.

Once it works, set another environment var: SONAR_RUNNER_HOME to the root installation folder for sonar-runner. (for example: C:/sonar-runner/) This is used when you want to run the sonar runner from a folder other than the /bin folder. (ie: when you run sonar-runner from your source folder).

Change the conf/sonar-runner.properties file. This has general configuration items used by the runner, some of which can be overridden in the project's config file (we'll get to that later). At a minimum, you'll need to set the sonar.host.url to point to where you have your sonar server running, and the database config. For the db, just copy/paste the lines from the sonar.properties file you created in Step 2. Again, example values are provided in the file (commented out) which will likely work just fine. Don't forget to comment out the Derby lines if they exist.

Step 5: The C# Plugins

You'll need to install the "C# Plugins Ecosystem" (ie: The plugins to
analyze C# code). Login to the SonarQube website as an Admin user and click the "Settings" link in the upper right-hand corner,
then click "Update Center" in the left-hand navigation pane. You should see an "Available Plugins" tab, under which you will find
all of the officially supported plugins. Search down the list to find "C#" under the Additional Languages section and click on the
"C#" link. You should now see an "Install" button -- click it!

Under the hood, this is installing each of the C# plugins .jar files into the /extensions/plugins folder of your SonarQube server
installation folder. You will need to restart the SonarQube service after you place the files for them to be available.

The plugins consist of the following: (as of this writing, which uses v2.1 of the plugins)

sonar-dotnet-plugin - .NET Core (support for the .NET-based languages - general API used by other plugins -- required)

If you're going to use StyleCop, you can either use a bundled version included with SonarQube, or install your own.

If you're going to use Gendarme, you can either use a bundled version or install your own.

If you're going to use Gallio, you'll need to install it as well as
OpenCover (or you can use NCover, if you have a license). When installing OpenCover,
I suggest using the Advanced mode and setting it to install for all users. Also note: Gallio does not support NUnit v2.5.4 or higher,
so you’ll either need to update the Gallio plugin
or jump to NUnit 2.6 (or other supported version).

Step 6: The sonar-project.properties File

Each Solution will need to have it's own sonar-project.properties file. This file will need to exist in the folder from which
you execute the sonar-runner. To make this easy, I would suggest putting the file in the same folder as your .sln file.

The file will have a few sections, which I will describe here.

Important Note: Any folder names in the config file will need to either escape the backslash with another backslash (\\)
or use a forward slash (/). I've chosen the latter.

Project Identification:

This section will provide the project key used by the SonarQube server to group analysis results over time, as well as provide a
useful name in the UI, etc. This should be unique across projects. The project version can be used to track different branches, etc.

Then, describe the source code layout. The "sources" field points to the top-level folder where the source code exists. If you're
.sln and .csproj files have relative paths internally, then this should be the top-level folder. Assuming you don't have any strange
layouts, this will likely be the same folder as your .sln file (which is likely where your .properties file exists), so can just
be ".". Additionally, you need to denote that the language is C# using the "cs" value.

# Info required for Sonar
sonar.sources=.
sonar.language=cs

C#-specific settings:

Here, you'll need to provide information about where the .sln file is located and where key libraries are located, and which version
of .Net you're using.

#Core C# Settings
sonar.dotnet.visualstudio.solution.file=DemoApp.sln
sonar.silverlight.4.mscorlib.location=C:/Program Files (x86)/Reference Assemblies/Microsoft/Framework/Silverlight/v4.0
sonar.dotnet.excludeGeneratedCode=true
sonar.dotnet.4.0.sdk.directory=C:/Windows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0
# To prevent any issues while analyzing multiple solutions containing projects with similar keys
# Will be set by default to safe starting at version 2.2: http://jira.codehaus.org/browse/SONARDOTNT-339
sonar.dotnet.key.generation.strategy=safe

Plug-in Specific Sections:

For each plugin, there is a "mode" setting. If blank, then the plugin will run. If you want to skip/not run a plugin, set the mode
to "skip".

For Gallio, you can stipulate if you want to use OpenCover (free) or NCover (not free). You can also stipulate the runner mode.
I had trouble using anything other than "Local". You will also need to stipulate the naming pattern (regular expressions, I believe)
for the Visual Studio projects that include unit tests. You can have multiple patterns, separated by semicolons.

Step 7: Running an Analysis

From the folder with the sonar-project.properties file, run the command $SONAR_RUNNER_HOME/bin/sonar-runner

You'll see the runner start up, listing some details like the working folder, etc, then it will kick off the source code parsing,
then the plugins (such as running the unit tests). If there are any errors, a Java exception will be thrown. Sometimes these contain
enough details to troubleshoot, sometimes not, so you may need to run with the -X command line argument to get additional details
when errors occur.

One error I run into frequently is UTF-8 Byte Order Mark causing the source file parsing to fail. You will see this in two ways:
the files will not end up in the SonarQube web UI and something like the following will show in the runner log:

You can fix this by setting the encoding in your sonar-project.properties or the sonar-runner.properties file:

#----- Default source code encoding
sonar.sourceEncoding=UTF-8

After a successful run of the analysis, you can see the results in the SonarQube server webpage. Unless you have changed the port
in the sonar.properties file, this will be at http://localhost:9000/

The homepage will have a link for each project (the value provided as sonar.projectName), which will take you to the project
overview. Clicking on any of the links on the overview page will take you to the drill-down data for that metric.

The following is an excerpt from an email/blog I wrote for internal consumption by my development team, but I think the concepts are universal and worth sharing. I've removed any specific references to company intellectual property and otherwise "sanitized" the details, so some of the details are lost in the process.

For those unfamiliar with the term, Cyclomatic Complexity is a formula for determining the varying logical paths that can be taken though a piece of code, thus determining a quantitative value for the code's complexity. A higher Cyclomatic Complexity score means that there are a lot of code paths and/or factors that affect the logic with the code. One way to think about it is this: to fully unit test a method, you should have one unit test for each logic path - thus a score of 10 would result in as many unit tests. Wikipedia entry: http://en.wikipedia.org/wiki/Cyclomatic_complexity

The higher the CC score, the more difficult a piece of code is to read and understand, to test, and therefore, riskier to modify and maintain over time. A general rule-of-thumb suggests keeping the CC score to under 15 greatly improves readability, maintainability and reduces defects in the code. Anything above that, and the code is likely violating the single responsibility principle and is trying to do too much.

So why do I bring this up?

Because over the last 9 months as I've been learning the company's codebase and delving into many areas of code where I have no prior knowledge, I've come across sections of code which have been difficult to pick up and understand, and/or which requires and extraordinary amount of knowledge about the inner workings of that code and the code around it (tight coupling). And, as a direct corollary, this code is has low (or no) unit test coverage to protect us against mistakes and misunderstandings.

From theory to concrete examples:

A little further down, I will discuss how I suggest we address this issue, but first I want to give some concrete examples of what I'm talking about.

I was recently tasked with making a fairly straightforward change to what data is displayed in one of our "legacy" controls; however, it took me almost a full day just to wrap my head around what the control was doing and all of the (inconsistent) ways other classes/controls interacted with it.

So what makes this method particularly problematic? Nested conditionals.
At one point, there's:

an if block,

with a nested if block

with a nested if block

with a nested foreach loop

with a nested if block

with a nested switch block

with a nested if block.

There are 11 if statements and 3 switch statements in a 96-line method -- 66 lines, if you don't count non-code lines.

Method: public void Initialize()
Cyclomatic Complexity: 17

This method does not have the level of conditionals of SelectAll(), but it does change behavior based on the type of a public property. In this case, the control accepts four types of DataSets (we utilize typed DataSets, which extend the DataSet class -- so there's a ProductDataSet, PricingPlanDataSet, etc) and acts differently based on the type of the dataset.

Another factor that affects code complexity is high levels of coupling with other classes, and in this case the Initalize() method is coupled with 36 other classes. This means that there is an increased potential that changes in those other classes will affect the logic in this method.

So, what can we, as developers, do?

Step 1: Knowing you have a problem

Aside from just generally being aware of Code Smells when you see them, there are a couple of tools that will help specifically call out Cyclomatic Complexity:

There is a free "CyclomaticComplexity PowerToy" ReSharper plugin that will mark any methods with a CC score above 17 with a ReSharper warning and display the CC score as a tooltip. It can be downloaded here: http://resharperpowertoys.codeplex.com/ (make sure to grab the v5.1 installer pack if you haven't upgraded from v5 -- the source code is already converted to use ReSharper v6). This will give you more "real time" warnings that things are ugly.

Additionally, Visual Studio 2010 Ultimate comes with code analytics features which will calculate several metrics for your code, including Cyclomatic Complexity, Depth of Inheritance, Class Coupling, Lines of Code, and something Microsoft calls a "Maintainability Index", where they try to apply an aggregation of the others to give you an overall score.

You can access this feature from the Analyze menu

This can be quite time consuming, though, depending on your hardware and the size of the Solution/Project you decide to analyze. At the end, VS2010 will spit out a report, which can be exported to Excel.

Step 2: Refactor to reduce complexity, improve understanding

I am a firm believer in a phase I picked up from Martin Fowler's writings (which he attributes to "Uncle Bob" Martin):

"always leave the code behind in a better state than you found it"

For me, this means small refactorings to address issues or improve understanding of the code. For example, if you're coming in to a piece of code and it takes you more than two minutes to figure out what the code is trying to do, you should look for ways to make the code more clear and embed the understanding you just gained into the code. Maybe that means pulling a set of lines out into their own method and naming that method in a way that makes it clear what those lines are doing. In other cases, it may be a larger refactoring effort.

The key here is make it better than you found it -- it doesn't have to be perfect. If everyone who touched the code makes it better each time, then those areas of the code that get touched a lot improve quickly -- which is good, because areas that change a lot are the places that have the most risk of getting broken.

So in my examples above, I would apply this rule in two ways:

pull out those nested conditionals in SelectAll() into sub-methods (assuming there isn't a clear way to remove them altogether)

utilize inheritance to reduce conditional blocks, such as those in Initialize(). By having the core/shared functionality in a base class, then having derived classes for each of the DataSet types, we could move the type-specific code into easier-to-maintain and smaller classes and greatly improve the readability and maintainability of the code.

Now, obviously I didn't do these things, otherwise they wouldn't still be here as examples. Why not? Well, because this class is tied to core functionality in our system and zero unit tests, so we would need considerable QA effort to regression test this functionality -- and had no QA resources available to do that. So from a risk standpoint, this was not the time to make changes. However, I do hope we can get scheduled time in the (near) future to do planned refactoring of this code.

Step 3: Don't make things worse

This one's easy: When writing new code, make sure it's clean, readable and testable.