Java development with an iPhone touch pad for the Atari 2600 from an urban hip-hop perspective

If you try to fail, and succeed, which have you done?

From T.D.D. training two weeks ago I learned that failing is just as important as passing. More specifically certain tests are expected to fail and you need to verify that they do. It’s a little counter intuitive but think about it. The first kind of failing test that deserves our attention is the initial test. The first time you write a test it should fail and you should verify that it does. That ensures that you are actually adding the intended behavior when you finally do get a green bar.

Another type of important failure is the resilience test. These are test that check your code’s resilience to bad input. Lets just pretend that you have this:

Here we are exercising Hiro’s ability to teleport to other planets which we would expect to fail. (It should fail because there’s no oxygen on Mars which would in effect kill our Hiro or Hero, if you will.) We want our Hiro to throw an Exception on inter-planetary travel… something like InterplanetaryException. This is a situation where we try to fail. If you run the test as is and it green-bars what have we done? Have we succeeded or have we failed? It’s a really mind-twisting bit of code the first time you write such a test because you have to think opposite then opposite of that opposite then rationalize the intent of double negatives and all.

So how do we assert an Exception in JUnit? (It turns out that the folks at CxxUnit have done a much better job at the task.) First you have to execute the code that you’d expect to throw the Exception as above then you have check that the exception is thrown. The easy way to do that is to put another line of code after the line you’re testing, an “if-we-get-here” line. Because if we get that far then the Exception hasn’t happened which means we’ve failed.

We run the above and get our red bar which tells us to change/add the logic necessary to trap for bad input. Assuming the createTeleporter() method just returns “new HiroNakamura()” we’d do something like this.

Now we’re in big trouble. It is so typical to make the above mistake then live with the consequences days or months later. (If you know where the problem is keep quiet so the people in the back can concentrate.) What happens if later on somebody removes the guard clause in HiroNakamura? Or what happens when we create a new Teleporter implementation (PeterPatrelli) and attempt to reuse the above test logic? Let’s consider the first case where somebody thinks that the Planet object would be necessary for qualifying a location. In other words we want to be able to teleport to “Planet earth = new Planet(Planet.EARTH)” after we’ve set a country on it like, “earth.setCountry(“Japan”);”. So we remove the check at the beginning of Hiro’s teleportTo method and the test passes! That’s no good because now we can also send Hiro to Pluto and freeze him to death. The problem comes from the fact that we set out to fail and succeeded in failing which gave us a false sense of success. (That makes sense, right?) Instead should we have set out to succeed in failing and properly asserted our failure for success? Let’s look at our test.

Why is it no longer failing when the Exception doesn’t happen? What’s really going on? Well we teleport and then we run the “we-shouldn’t-get-this-far” line and that should fail the test right? Well it would fail the test because it throws an AssertionFailedError which we blindly catch as Exception and silence with our naive comment that states stoopidly that we expect an Exception. (Never ever EVER catch or expect Exception!) Instead we should have caught the right type of exception which would not only self document the intent of the desired behavior but would have kept us safe from future modifications.

Post navigation

4 thoughts on “If you try to fail, and succeed, which have you done?”

You probably know this but in junit 4 you could write your test like this, which would fail if the exception isn’t thrown. I believe there is a way in junit3, using a exception suite or something from the extensions package to do something similar.

Yeah I knew that JUnit4 and even TestNG have a nicer way to handle the issue. Even CxxTest has exception assertions. The point of the post is to stress an important and often overlooked step in TDD… the red bar. I also wanted to stress the importance of the concepts of making sure the code you write both in the test and in the tested object is necessary and correct. Thanx for the compliments y’all.