Search QALab

Project Documentation

Legend

Quick News

2006-11-30, if you intend on using QALab with Maven 2, please do read the plugin FAQ, thanks.

2006-11-27, QALab is integrated in JBuilder 2007 as part of the new TeamInsight team-RAD feature.

"We are really excited at the collaboration between ObjectLab and CodeGear"
said Joe McGlynn, Product Manager for JBuilder. "Our engineers were searching
for an innovative way to consolidate and trend key project metrics and we
found a truly wonderful solution in QALab".

ObjectLab has worked with Borland's Developer Tools Group ("DevCo/CodeGear") to make
the transition to Maven 2 smooth and rich 'out-of-the-box'. QALab itself
will remain independent and carry on supporting all platforms

2006-11-12, version 1.0 is out! After over 3 years of internal use and 15 months as an open source project, we finally ship a version
that we consider good enough to be named '1.0'. We are very proud of this little project, it has generated a fair amount of press. We would
like to thank our colleagues, friends and users who have helped us reaching this stage. We will also try to help and improve the product and we already
have a few more ideas... and some very interesting news coming soon...Watch this space!

QALab Introduction

QA Tools like checkstyle, pmd, pmd-cpd, findbugs, cobertura (cobertura-branch and cobertura-line) and simian are great build tools but they only take a
snapshot of the state of your project. You do not get a sense of the trend of your project.

This project collects and consolidates data from several QA tools and
keeps track of them overtime. This allows developers, architects and project managers alike to be presented
with a trend of the QA statistics of their project.

The following tools are currently supported:

Checkstyle: code style validation and design checks. QALab keeps track of number of violations per file and overall.

PMD CPD: Duplicate code (always a bad idea) detection. QALab keeps track of number of the overall number of duplicated lines.

FindBugs: fantastic tool to detect potential bugs (really!). QALab keeps track of number of violations per file and overall.

Cobertura: Coverage tool. QALab keeps track of percentage of branch and line coverage.

Simian: excellent duplicate code detection (non-open source). QALab keeps track of number of the overall number of duplicated lines.

This project can be used via ant or Maven. There are three main steps to using QALab:

Collect Data from QA Tools into a generic qalab.xml format. This is a necessary step to consolidate data.
Refer to the documentation from Checkstyle, PMD, PMD CPD (Copy Paste Detector), FindBugs, Cobertura and Simian in order to generate the xml reports.

Generate Charts from qalab.xml for each file; these will show the trends over time for each file and the
overall project.

Generate a summary xml and html pages with the files that have seen a change in their QA statistics over the last n days.
This is particularly useful for developers who can see the impact of the latest code checked in.

Here is a highlight of the steps available, in green are the input files, generated by the respective QATools. In
yellow, the files generated by QALab and each transition is a task in QALab.

Data Consolidation

PMD, Checkstyle, Cobertura, PMD-CPD, Simian and FindBugs have each their own XML format. This task is to
extract some information from each file, typically the number of violations per file and typically to
add this information to a qalab.xml file. QALab is gathering statistics info and therefore, we are
not interested in the exact 'violation' or error (see checkstyle, pmd, etc for that).

Please note that the qalab.xml will keep growing and should be kept in your
source repository or backed up regularly. The more info you have in it, the more valuable it becomes.
As such, it is recommended to use QALab only say, once a day when the whole set of checkstyle, pmd, pmd-cpd,
Findbugs and Simian statistics are generated.

The QALab team realise that some project may generate very large qalab.xml files if they have lots of
classes and lots of violations. QALab was used for over 2 years on a 450,000 lines of code project without
any issue. Having said that, QALab offers you the ability to define how to save
the consolidated statistics; simply implement the QALabExporter interface, say to a DB?
Most of the examples are shown with the XML file.

In its default configuration, QALab will will create a new summary and result entry for each run and file
each time you run the 'merge' task. This can be controlled by using a given timestamp (or forcing it to
use only a 'date') and an action 'replace', in which case, each run of QALab 'merge' will replace the
previous one with identical timestamp and type. This is very useful if you want to run merge several times per
day but want to keep only the latest set of stats per day.

The XML format uses primarily attributes as it tries to limit the number of elements in order to save
some space. The name of the attributes though is quite explicit.

Please note that Cobertura generates 2 types of statistics in QALab: cobertura-line and cobertura-branch,
corresponding to the line and branch stats in the coverage.xml file

Generating Charts

Summary charts for all or a subset of statistics types (checkstyle|findbugs|pmd|pmd-cpd|simian|all)

Charts for each file, typically a class, with a statistics and then again, you can select the set of statistics)

Generating Movers and Shakers Summary

As a project grows and the number of developers increases or changes, it is becoming more and more
difficult to keep track of the changes impacting the code quality. The best way to deal with this is
to deal with any deviation as early as possible. Some style violations may remain in your code for months
and are accepted as such.

This report will only show the changes over a given time window, it will highlight the Up (baaad!) or Down
impacts (good!) on the code.