NBench Performance Testing – NUnit and ReSharper Integration

All the logic to run performance tests in NBench is actually embedded in the core library and for this reason an external runner is not strictly required. You can easily integrate your unit testing framework and use your favorite tools to run your performance tests.

In this post, I show you how you can run NBench tests with NUnit and ReSharper.

How to create your NBench performance tests

You can create performance tests using attributes as usual.

The only difference is to inherit from a new base class.

The PerformanceTestSuite base class

The abstract class PerformanceTestStuite<T> defines a parametric test in NUnit that run a single performance test (benchmark). The TestCaseSource attribute point to a method that returns all the performance tests so this single NUnit test basically runs all the performance tests defined in the type T.

The Benchmarks method use an instance of the ReflectionDiscovery class to find all the benchmarks and then return a TestCaseData for each benchmark found.

The discovery class takes an IBenchmarkOutput instance that is used by the benchmark.Finish() method to notify the result of the performance test. For each benchmark NBench tells us all the assertion results and we can use those information to run a NUnit assert and print useful information to the console.

Note: the reason why I had to use generics is because NUnit requires the TestCaseSource method to be static and in a static method I don’t have a way to get type information.

Running the tests in ReSharper

This is how running the tests looks like in the ReSharper test runner.

For a failing test, you get a clear error message.

For a passing test, you still get information about the actual results of your performance tests.

Conclusions

I think this is great! You can add this base class into your own project and very easily get your team to start writing automatic performance tests. Of course, the biggest problem is to define what to test but it’s great that now we have some tools that helps us to do it so easily. The learning curve is so slow.

NBench vision is to become a complete, user-friendly and fully extensible framework for automatic performance testing in .NET and more importantly, to make automatic performance testing a regular software development practice. I think this framework is a very good step forward in this direction.

Previous posts

I am a Software Engineer in Red Gate Software. My peers describe me as really enthusiastic, motivated with an infinite passion for software development, a relentless desire to keep learning and an amazing ability to involve others in my pursuit of excellence. I am a proficient C# developer and founder of DotNetCambridge: the official community on .NET in Cambridge.

You can easily adapt the super class to use xUnit as well. I can create a blog post about it. I created this class so it’s not part of NBench yet and there are many ways it can be improved. For example at the moment is picking measurements as well even if they probably should not. It’s a kind of an hack to integrate NBench with a unit test framework to proof that it is possible today. The idea of having separate package for NUnit or xUnit is not too bad, you can suggest it in the github project for consideration. Thanks