Tags

Visualizing PHPUnit runs

LESS, a project I'm currently hacking
on @work, has a unit test suite that was a pain to use because
the ~80 tests took 45 seconds to run.
The problem with that test suite is that you just don't run it often
enough during development because it's just too long.

Running the specific test cases for the part of the application
you're working on is easy and fast, but that does not tell you when
changes in part A of the app break part B - which happened several times,
and all just because I didn't want to wait 45 seconds again and again.

So a solution was badly needed; tests should be as fast as possible;
preferably < 10 seconds.
Before being able to make the slow tests faster, I had to find out
which of the 80 tests were slow.

Finding timing information

Now when you're using a CI
system like CruiseControl or Jenkins, you do of course have nice
reports that tell you which tests get executed, where it fails and what
time they took to run.
The normal PHPUnit output does not show any timing information,
and manually grepping the output of the --log-junit file
- which contains timing information - wasn't
something that I want to do if I can get pretty reports.

I don't have CI for LESS yet, so I needed some tool that generated
pretty reports with timing details instantly on the command line.

Reporting tools

All web searches for "phpunit reporting" led to pages that talk about
coverage reports, but not a single one told me how to visualize normal
phpunit test runs without coverage.
The human readable
normal output
formats do not show timing information.

Since PHPUnit supports
JUnit
log file output, I thought that I also could use some JUnit
reporting system.
It turned out that there are some commercial ones that generate pretty
PDFs, but I didn't need anything too fancy (and closed source, and
I didn't want to buy anything).
Only relatively late I stumbled across the
antjunitreport task which
promised to do what I wanted.

Reason for it was that I had the redland PHP extension
enabled; it
adds restrictions to XSLT. Disabling it made report generation work.

Output

It supports a framed and a single page version (noframes).
Here some screenshots:

Single page view:

Results

Now that I had the report and saw which test suites took most time,
I could identify seven test methods that took about 5 seconds each.
They used an RDF
file to get example data - but that file was 1.1MiB huge.
Parsing it took most of the time.

After replacing the file with a customized version that contains
only the tags needed, all tests run at < 0.1 seconds now.
Running the whole suite takes 10 seconds, and running all tests
is fast enough now to do it again and again.