Unit tests usually run a piece of code to verify the output or state to ensure it does what you excepted. With exceptions, it gets trickier. Once one is thrown the test ends abruptly, so how can you make sure that it was really triggered?

To demonstrate various strategies for testing exceptions, I’ve made a small example project in the form of a simple calculator. Most of the tests use plain JUnit4, except one which takes advantage of AssertJ assertions. But before we look at that, we should clarify what we want to accomplish by testing for exceptions. As with all testing, the main goal is to verify the code does what it is supposed to. In this case; throw an exception given a certain state or input. So we want to verify three things:
1. An exception was thrown
2. It was triggered by the state or input we wish to test
3. It was the error we expected

The example calculator is capable of adding or subtracting numbers. We will ignore the implementation for now, assume it is sane and focus on the tests. It has a special rule though, it should only add positive numbers together. For negative numbers, corresponding subtraction should be used instead. So if anything fails to follow this business rule, I want the the add()-method to throw an exception.

The first approach is covered in CalculatorAnnotationTest.java. This suite contains some normal tests to ensure the calculator works as intended and one to verify it throws an exception when adding negative numbers. The latter is annotated with @Test(expected = IllegalArgumentException.class). This tells the test runner that the test should throw an exception of the specified type. The main problem here is that the annotation covers the whole test, which means that if any line throws such an exception the test will still pass. If we try to comment out the last line with calculator.add(1, -1); we might expect the test to fail since its no longer adding anything, but to our surprise it is still passing! Sounds like something else in the test is triggering an exception, but it’s hard to tell since it doesn’t seem possible to verify error message we get with the annotation. Thus, it only succeeds in point 1, but fails on 2 and 3. As soon as you do more that one thing in a test, you can no longer be sure which of the statements triggered the exception.

Since the annotation seemed too broad, let’s try to focus more on what we are trying to test. Ultimately, we want to know if the statement calculator.add(1, -1); throws an exception. So how do we normally deal with exceptions? Try-catch, of course. On to CalculatorTryCatchTest.java. This is a quite normal pattern which has several variations, but the core concept is that we do some set up, then call the statement we wish to test inside a try block and assert that we got the exception we wanted. Of course we also need to keep an eye out for other possible outcomes, so there’s two additional checks to mark the test as a failure if another or no exceptions are thrown. Without, the test would still pass even though it didn’t trigger the exception we want.

When running this test it is easier to see why the annotated test failed earlier; the constructor is rather picky and excpects the name to start with a capital letter. Once we’ve fixed that, the test works as expected. Actually, the constructor should be called outside the try-block, since it is only part of the arrangement setting up the necessary prerequisites for the test. The core of the test is the add()-method, so we should have as little as possible else inside the try-catch. If an exception is thrown in the setup, the test should of course fail because we didn’t achieve the necessary state to test our specification.

This way, we know the exception was thrown, we’ve limited the code in the try-block to just the method call we want to test and we inspect the error message to verify what we got. In other words, the try-catch pattern accomplishes all three goals we established at the start. However, it is a bit cumbersome to set up with all the try-catch boilerplate each time we want to test an exception. Worst case, we create an incomplete test which misses a case without reporting the test as failing.

For an alternative use of this pattern, see CalculatorTryCatchAlternativeTest.java. It uses a boolean flag to ensure that we don’t leave the test without asserting properly. I think this is somewhat better, but we still need to write a lot of boilerplate and end up introducing a new variable.

While I think the try-catch pattern is a step in the right direction, I’m not too happy with the need to add the same extra lines as safeguards over and over. Luckily, I found that JUnit (version 4.7 and newer) comes with a built-in rule to make this easier. The rule is called ExpectedException and is used in CalculatorExpectedExceptionTest.java. The rule is defined at the top and basically says that by default, no exceptions should be thrown by the tests. But where you want an exception to be triggered, the rule can be instructed to look for the exception type and error message which is expected. These instructions can be placed after all the all the setup so that we have it separated from the minimal section we wish to test.

This guarantees that the exception is triggered where we wanted it (goal #2), as well as specifying type and error message (goal #3). If it doesn’t encounter an exception matching its expected criteria it will mark the test as a failure, thereby fulfilling goal #1. All in all, it does an excellent job to fulfil all requirements.

The examples above are all using JUnit4, but I also looked for other solutions. I found AssertJ, a fluent assertion framework which contains a lot of useful things. (It started out as a fork of fest assert, for those more familiar with that). CalculatorAssertJTest.java contains an example demonstrating how it can deal with exceptions.

The code which should throw the exception is placed inside a lambda expression, which makes it possible to observe and verify the result from the outside. In terms of separation, this is perhaps one step further than the other examples, since we know that the exception can only be triggered by the code we place inside the lambda. We can also inspect it, looking at the type and the error message. This allows us full control to verify the exception when it has been thrown as well as cleary separate the section we except to throw something from the other parts of the test.

In conclusion, I prefer ExpectedException because it gives you the greatest amount of control/readability when testing exceptions. The annotation can lead to brittle tests if they have more than one line or method call in them. Setting up try-catch each time seems too cumbersome, plus I fear it is far too easy to write a bad test if you forget to add one of the safe guards. I liked the AssertJ approach though, I will consider using this for future projects.

As the observant reader might notice, there’s currently no way to add a comment at the end of this post. In fact, it’s not possible to add comments to any of the posts. What’s going on?

The comment section was initially enabled to facilitate feedback and discussion. In practice there hasn’t been any of either. There has been plenty of comments though, which I’ve had to mark as spam from time to time. Recently, there’s been a sharp increase and since I don’t get any interesting comments I found it preferable to just disable comments all together.

Thus, the comment section has been disabled for the time being. It may return in the future, but I would need to find a better solution that the current one.

I recently finished reading The First Law trilogy by Joe Abercombie, which consists of “The Blade Itself”, “Before They Are Hanged” and “Last Argument of Kings”. While a fantasy series, it is also part of the grimdark subgenre. As can be guessed by the name, grimdark is darker and grittier than “normal” fantasy. Rather than a classical good versus evil story told with clear black and white characters, they characters come in varying tones of grey. It is comparable to George RR Martin’s Song of Ice and Fire (also known as Game of Thrones) where there’s really no clear-cut good guys.

This is evident in one of the main characters in the series, Inquisitor Glokta. He used to be an officer, but after being captured and tortured in a war, he has now turned to torturing others. I’ve seen him compared to Black Adder in other reviews, and while I don’t fully agree in this, I can certainly see the similarites. I would rather compare him with Dr. House, since he’s smart and capable at what he does, though constantly in pain. Glokta easily has some of the best lines in the books, and his inner monologues are a thrill to follow. He especially shines in the second book, “Before They Are Hanged”, where he is tasked with running a city while investigating why his predecessor vanished. Oh, and and the city is besieged by an army much stronger than any defence they might be able to put up.

The two other main characters are Logen Ninefingers and Jezal dan Luthar. Luthar is a young officer which is training for the annual fencing contest, hoping to win fame and glory. While busy practicing and spending his evenings playing cards, he is eventually dragged into a quest for an object which might change the fate of the world. Logen is a barbarian from the north which has been a warrior for most of his life. In addition to his skill in battle, he is able to summon and talk to spirits. After being separated from his group of fighters and assuming they have perished, he heads south. Shortly after, he is called upon by Bayaz, the First Magus, which has use for someone who can talk to the spirits.

Bayaz is a powerful wizard, who has played a vital part at several times throughout the history of the world. The backstory is presented through various means (including a play!), and helps both explain what has happened earlier and show how historical events affect the present. He is a wise old man, but can also be intimidating in his displays of magical power. I find it interesting how he fills a similar role to Gandalf, yet does things which Gandalf would never do. This is one of the fun things in the books, how the author plays with the preconceptions of how the story will progress. An example of this is that the very first chapter literally ends with a cliff-hanger.

All in all, quite interesting books. Abercombie’s also written some standalones which take place in the same universe, which I look forward to checking out.

By popular demand, my thoughts on the final volume in the Wheel of Time series; A Memory of Light. With 14 books and one prequel novel in total, it is one of the longest fantasy series and it has now come to an end. When Robert Jordan passed away, a lot of us were worried we would never know what happened to the group of friends which had to flee their home town all these years ago, but now the last book is out. And so, more than a decade after I picked up the first book, I’ve finished the last one.

Being the final book in a long series, there is a lot happening. Factions clash, prophecies are fulfilled and the fate of the world is determined. It all culminates in Tarmon Gai’don, the Final Battle, which gets a nearly 200-pages long chapter dedicated to it. Much of the series have been leading up to this event and most of the plot threads are resolved, as well as some new mysteries introduced. (Those who have read it know what, or rather who, I’m talking about).

It was really nice to see the series finished, and I think Sanderson did a great job wrapping it up. It is one of the best series I have read, which is a bit ironic since I initially gave up on it merely a chapter or two in. When I picked it up a second time though, I couldn’t figure out why I had abandoned it. I really enjoyed the characters, the varied cultures they encounter in different parts of the world, the magic system and the glimpses into the lost glory and wonders of the Age of Legends.