2011-11-02

Silverlight Unit Testing: TestInitialize/TestCleanup methods do not fire, tests do not run, nor pass/fail, and no details given

I had the classic bang-head-against-the-keyboard for three hours scenario today while doing Silverlight Unit Testing.

I had a test class with two unit tests in them which worked fine, until I started refactoring to get common setup/teardown functionality moved into methods with TestInitialize/TestCleanup attributes applied.

Then the weirdness started. I started seeing this:

The tests don't run; don't pass or fail; and no details as to why.
Check out the yellow highlighted areas.

As you can see, there wasn't a whole lot of information present to assist me in my predicament. After removing the TestInitialize/TestCleanup attributes, the tests worked fine.

I began searching the internet trying to find someone who ran into the same problem. I found many results where developers complained of the TestInitialize methods swallowing exceptions and moving on, so I started down that path in my own code, trying to discover if there was an exception I was missing that was being eaten and causing the entire class to fail.

The only problem was, when I commented all code within the bodies of the TestInitialize/TestCleanup methods, I still ended up with the same results.

???

Digging deeper, it finally dawned on me that the test class in question was inheriting from a base test class that also happened to have methods dressed with the TestInitialize/TestCleanup attributes. Looking at the declaration of TestInitializeAttribute revealed what I suspected:

AllowMultiple = false... which makes sense. So my code had two problems:

My base class' method for TestInitialize needed to be made virtual so it could be overridden in classes that inherit from it to allow for overriding (and calling back to the base so both get their jobs done.)

My child class' method was named different, so I needed to go make it match and override.

Once I fixed these problems for both the init and cleanup methods, the tests went back to working accurately.

Bottom Line: When doing unit testing with base test classes that perform common setup functionality, make sure to have your Setup/Teardown or Initialize/Cleanup methods marked virtual in the base class, so they can be overridden in the sub-classes. Then override those methods in the sub-class, making sure to call back to the base appropriately.

Example (MSTest):

public class CommonTestSetupBase
{
[TestInitialize]
public virtual void TestSetup()
{
// do common setup work for all that inherit
}
[TestCleanup]
public virtual void TestTeardown()
{
// do common cleanup work for all that inherit
}
}
//=================
[TestClass]
public class UnitTestedClassTests : CommonTestSetupBase
{
[TestInitialize]
public override void TestSetup()
{
base.TestSetup();
// do common setup for this class
}
[TestCleanup]
public override void TestTeardown()
{
// do common cleanup for this class
base.TestTeardown();
}
}
I knew I wouldn't remember this if I bump into it again, so I had to document it here... I wish I had more time to go into more detail, but I hope this will be enough to help you and I in the future when we meet this problem again.