I Want Assert.NotYetImplemented(); In .NET Test Frameworks

I do that a lot with c# and nunit. It helps me to figure out where I need to go, what tests are going to be organized where, etc. I certainly don’t expect my first pass at the tests to be the final layout, and I don’t try and get every last test laid out all at once. I just lay down the ones that I know of immediately. I also tend to add new test shells as I am working in the tests and implementation, as I think of scenarios. This gives me a good place to start and a overview of what the code should be doing at a point right in between the user story and the implementation. Leaving the tests failing (with a “throw NotImplementedException()” or an “Assert.Fail(“not yet implemented”)”) gives me the freedom to go about my business and focus in on what i should be doing next, without having to worry about keeping track of the things i will be doing in the future. It’s quite nice, really. I’ve been doing this for a while now, and it works for me.

But… here’s the thing that started bothering me this morning: if I were writing ruby instead of c#, there’s no way I would leave failing tests lying around like this. Worse yet, if I were writing code in collaboration with any of my coworkers, I would not want to burden them with these failing tests. After all, if I commit these test to source control, then I’ll fail the build and that’s definitely not what I want to do. So, instead, I find myself keeping my code locally (in git, of course… local source control safety) and not sharing with anyone else until I’m done with the tests that I laid out.

This is counter to the collaborative, incremental nature of software development that we should be employing. The only time I should horde my code on my local box is when I am in the middle of working on something and it is obviously broken. Otherwise, I should be pushing my changes into the shared source control whenever I have parts of it working.

Taking A Lesson From RSpec

If I were writing ruby and rspec tests, I wouldn’t have this problem. I could simply omit the code block for my tests like this:

1: describe "when something is happening here"do

2: it "should do this thing"

3:

4: it "should do that thing"

5:

6: it "should do some other thing"

7: end

by leaving out the “do … end” code blocks in each of the “it” tests, rspec will gracefully produce the type of results that allow me to have my not yet implemented tests, but still share my code and my tests with the rest of the team:

The tests come back as a nice yellow color with a message stating that they are not yet implemented. It is easy for me to see where I should be working next, yet still commit this code to the shared source control and have other team members and the build server run without being clobbered by me failing tests.

I Want Assert.NotYetImplemented

That’s what I want… I want the ability to have a not yet implemented test, like rspec. It should not be too terribly difficult for the Nunit, xUnit, MBUnit, MSpec, and other test framework crews to add in this code. Something as simple as this would work:

1: [Test]

2:publicvoid some_test_that_i_will_care_about_soon()

3: {

4: Assert.NotYetImplemented();

5: }

And then we can get the CI build servers like CCNet, Hudson, TeamCity, etc, and the test runners like Resharper, Gallio and TestDriven.NET to join in and update how they report tests. It will take time, of course… there are a lot of players in the .NET community that would need to coordinate this type of change… but it’s worth it. It gives us a lot more flexibility in how we operate in our teams.

Have you considered Assert.Inconclusive(“Not yet implemented”)? This should give you the nice yellow test result you seek, and should work with existing CI software too. It’s definitely in NUnit 2.5, maybe 2.4 also.

IgnoreAttribute is also a good option, as other commenters have mentioned. There may be important semantic differences between Assert.Inconclusive() and IgnoreAttribute that I’ve missed.

Harry Steinhilber

MSpec already has this functionality. Simply declare but don’t assign the It delegate:

It should_do_something_I_will_care_about_soon;

This will generate a not implemented message very similar to how RSpec does it.

Dan

Could you create an Assert extention method that just called Assert.Ignore() ?

Dale

There is also the option of making the class or method under test throw a NotImplementedException and expect that exception in your test.

to the peeps that suggested Assert.Fail or throwing a NotImplementedException: you might want to read the blog post you are commenting on. :)

to everyone else: good suggestions! I’m not a fan of Ignore, honestly. i think the semantics between ignore and not yet implemented are significant, but that would at least work. i’ll have to try out the Assert.Inconclusive, too. I was not aware of that assertion.