Connect

Connect with me

Friday, September 18, 2009

Comparing Unit Testing Framework

Introduction

I have been doing Unit Testing for past 18 months or so and have been ripping benefits of the same. I would suggest every developer should adapt unit testing methods if they have not done so far. Because we follow Agile in our development activities its mandatory for us to unit test our code. But even otherwise I feel its a disciplined effort to test your code. I have personally used NUnit as well as MS Test frameworks for unit testing and would like to highlight some differences here in this post.

Unit Testing Framework comparison

Recently I conducted a training session for the people in my organization who were interested in implementing it in their future projects. One of the feedback I had received during my previous training program was that I should have used NUnit for demo instead of MS Test because people felt that unit testing without NUnit was a crime. Just kidding. It was just that NUnit has become so synonymous with unit testing that people think its not worth attending a training if you are not going to demo NUnit.

I personally don't find much difference between NUnit and MS test. Both use similar concepts with the exception of attributes. Following are some of the attributes which I have used during unit tests written in those frameworks

NUnit

MS Test

SetUp

TestInitialize

TearDown

Cleanup

TestFixture

TestClass

Test

TestMethod

Many purists feel that Microsoft has copied the NUnit framework and added their own attributes. Only thing that MS Test has advantage over NUnit is that it integrates nicely into the Visual studio 2005 / 2008 IDE. It also provides the additional feature of Code Coverage. A jazzy UI is a good way to impress people to use MS Test.

Another advantage I find with MS Test is the generic syntax for the Asserts. We could specify the data type that we want to check while checking for methods like AreSame or AreEqual. Instead of checking objects, we could specify the exact type, for e.g. Assert.AreSame<string>(expected, actual, "Expected and actual values are not same);

One thing which I find irritating with MS Test though is the speed of execution of unit tests. Its comparatively very slow. Just to measure the performance, I ran the same set of tests both with MS Test and NUnit. To my surprise NUnit was at least 3-4 times faster than MS Test. I found a link which explains why is it so.

Conclusion

Given the fact that NUnit is open source there are more possibilities of it being updated with latest changes to frameworks and related stuff. Whereas MS Test being part of Visual Studio, we need to wait for major releases to get updates for it. In order to overcome the Code Coverage feature of MS Test we could use the TestDriven.Net tool which is an add-in for Visual Studio. It provides code coverage with the help on NCover. Recent versions of NUnit also support the concept of Fluent Interface which allows constraints checking using more readable format. It helps developers write tests and specially asserts in a more natural way.

Along with the two frameworks that I have compared here, there is also a recent addition to this list in the for of XUnit. The learning curve is slightly higher in this case as it doesn't follow the same approach to unit testing as that of NUnit or MS Test. It is still in its nascent stages but someone wanting to test pure .Net code can opt to use it provided there is enough time available to learn new syntax and concepts like Facts.

BankAccount.zipis the solution file which I used to compare the MS test and NUnit.