Continuous Testing

One of the practices widely adopted today is automated testing of one sort or another whether it’s plain old unit testing, tdd, bdd, or whatever, chances are you’ve heard of it, and more likely you’re either doing it or you’ve at least been exposed to it. The steps involved can vary depending on how rigourous your shop is such as writing a failing test, writing the code to make the test pass, and then refactoring both the code and the test etc, but in general you always end up writing some code and then clicking the button to run your tests and then acting on the results.

While this has and continues to work well for many people there are efforts underway to help developers work more fluently still by minimising the disruption in the development workflow and thus improving productivity. This comes in the form of a number of tools that fall under the moniker of continuous testing and I’ve been evaluating one just recently called NCrunch which, as it’s in beta is currently free. NCrunch takes advantage of multicore processors in your PC to automatically detect changes to code as you type. It then attempts to build the solution in the background and if it succeeds then proceeds to run your tests giving you immediate feedback without you even having to go and click the button to run them. In fact, you don’t even need to save your file! It just happens in the background all the time. Visually, it also displays a coloured dot next to each line of code. Green indicates the line was covered by a passing test, red shows the failing lines, whilst black indicates that the line is not currently covered by a test. Left clicking on a dot pops up a window showing you the tests that cover that line, whilst right clicking gives you options like ignoring or debugging tests.

In the short period of time I’ve tried this I’ve already seen how it makes a difference to my time within the IDE and it is definitely a good experience as long as you have a capable machine. It’s not so good on my virtualised Win7 installation on my dual-core MacBook (though workable) but works great on the same environment on a quad-core iMac so it should have no problems at all on a native install. One thing I’d like to see is some kind of summary window that expresses the code coverage as a percentage to get an overall feel for the project but that’s my only wish.

*** UPDATE *** v1.39b released 16/05/2012 now has a metrics window showing code coverage as a percentage!

NCrunch isn’t the only game in town however, I know Greg Young is behind another continuous testing tool called Mighty Moose which is built on top of an open source project called AutoTest.Net but I haven’t had chance to try either of those out yet. AutoTest, I believe, doesn’t run the tests until you save or build though.

Whichever one I end up going with I think it’s definitely something that will become a permanent part of my toolbox and find its way into my everyday workflow, as the immediate feedback is such a benefit. I’d recommend you head on over to the either the NCrunch or Mighty Moose sites and check out the videos to see for yourself.