Introduction

A lot has been written on the subject of test driven development, and especially on the idea that tests ought to be written first. This is an ideal for which I strive. However, I have a tendency to write the unit tests afterwards.

Some people learn better by example. This article, rather than going into great length about the principles of test driven development, will walk the reader through the process of building and testing an algorithm by writing the tests first, then changing the method being tested so that it
fulfills the tests.

The final code and all the unit tests can be found in the accompanying download. This will require NUnit and Visual Studio 2005.
The use of NUnit in this example is purely incidental, this example could just
have easily used the unit test framework fin Visual Studio Team System or any
number of other unit test frameworks. The example shows how to build a suite of
tests rather than how to use a unit test framework. The reader is expected to
understand the basics of using a unit test framework.

The specimen problem

I once saw a demo of how to create unit tests up front for a simple method. The method took a positive integer and turned it into roman numerals. So, I'm going to do something similar. I'm going to take an integer and turn it into words, in English. The rules for this may change depending on the language, so if English is not your only language, you may like to try to repeat this exercise in another language.

So, if the integer is 1, the result will be "one". If the integer is 23 the result will be "twenty three" and so on. Also note, I'll be using British or Commonwealth English. So, for 101 the result in words is "one hundred and one". In American English it would be "one hundred one".

The walk through

The algorithm will also be refined through refactoring techniques. Agile development methodologies, especially eXtreme Programming, suggests that you do the simplest thing possible to get the thing working. So, going by this premise, I'll work on the solution in bits. First, get it to return "one", then "one" or "two" depending on the input, and so on. Once 21 is reached it should become obvious where some refactoring can take place and so on. The final solution will work for 32 bit integers only.

Getting Started

Visual Studio 2005 has some features that can help with writing the tests first. A test can be written that calls into the class under test and the smart tags will prompt you with an offer to create the message stub for you.

The test should fail because the stub throws an exception, rather than do what the test expects.

NUnit reports the error like this:

"NumbersInWords.Test.EnglishTest.NumberToEnglishShouldReturnOne :
System.Exception :
The method or operation is not implemented.".

The next thing to do is to ensure that the code satisfies the demands of the unit test. Agile methodologies, such as XP, say that only the simplest change should be made to satisfy the current requirements. In that case the method being tested will be changed to look like this:

publicstaticstring NumberToEnglish(int number)
{
return"one";
}

At this point the unit tests are re-run and they all work out.

Test "two"

Since the overall requirement is that any integer should be translatable into words, the next test should test that 2 can be translated. The test looks like this:

The necessary code change starts to draw out a pattern. Of course, the pattern could have been quite easily predicted, but since this code is being built by the simplest change only rule, the pattern has to emerge before it can be acted upon.

The pattern repeats itself until it gets to 99. By this point the public method looks like this:

At this point it should be easy to see that some of the work that has been done previously can be re-used with a small amount of refactoring. First, refactor most of the body of the public method into a class called TranslateOneToNinetyNine. Then re-test to ensure that the refactoring process hasn't introduced any new problems.

In Visual Studio 2005, it is very easy to highlight some code and extract it into a new method, thus allowing it to be reused by being called from multiple places.

Now the public method looks like the following and all previously successful tests continue to be successful.

For numbers from 101 to 199 the pattern is "one hundred and X" where X is the result of the translation between 1 and 99. Because it would take too long to write all those tests, it is possible to write just the edge cases and one or two samples from the middle of the range. That should give enough confidence to continue onwards. In this case, the tests are for 101, 115, 155 and 199.

Test "two hundred"

From this point onwards the pattern of what needs to be done should be obvious. For that reason, the details of the intermediate steps are skipped until all positive integers can be written. If, however, you wish to read about these intermediate steps then you can read the unabridged version of this article on my website.

The limitations of an integer (Int32) mean that this section reaches the upper limits of 2147483647. Unless an Int64 is used there is no continuation to the trillion range.

Final stages

To this point, all positive integers are successfully being translated from an integer into a string of words. At this point, through code reuse, it should be a fairly simple matter to refactor the code to work with negative numbers and zero.

And the test passes. But what about that final edge case? int.MinValue? The test is written but it fails. The reason is that Math.Abs(int.MinValue) isn't possible. So, as this is a one off case, the easiest solution is to put in a special case into the public method:

// Special case for int.MinValue.
if (number == int.MinValue)
return"negative two billion one hundred and forty seven million " +
"four hundred and eighty three thousand six hundred and forty eight";

Conclusion

This article demonstrates how to unit test and build a piece of code in small increments. The tests continually prove that the developer is on the right path. As the code is built the constant rerunning of existing tests prove that any new enhancements do not break existing code. If, for any reason, it is later found that the code contains a bug, a test can easily be created that exercises that incorrect code and a fix is produced.

The full final code is available in the associated download along with a set of NUnit tests.

History

Version 1.0 of the article.

Note: This is the abridged version of the article. For the full article visit here

Version 1.01

A minor update regarding the expectations of the reader in the introduction.

License

This article, along with any associated source code and files, is licensed under The BSD License

Comments and Discussions

I just used this in VB. I'm a newbie and I did this:
<testmethod()>
Public Sub NumberToEnglishShouldReturnThreeAndFour()
Dim actual As String
Dim testVals = {"one", "two", "three", "four"}
For i = 1 To 4
actual = english.numbertoEnglish(i)
Assert.AreEqual(testVals(i - 1), actual, "Expected the result to be " & testVals(i - 1))
Next
End Sub

Then let VS stub out the class, made a case statement, caught 3 errors on the array (the -1 thing) and am bloody happy.

Total time from start to end-never done a unit test before OR TDD-was ten minutes.

Something I've been thinking about recently is the use of TDD with SQL Server. The use of TDD in applications normally seems to extend from the DAL onwards, but never actually extends to the database. For instance, it would be useful to be able to TDD a DTS package. If you have any experience/ideas, I would be interested to hear about it.

Databases can be quite difficult to test. Mainly this is because the test will often alter the state of the database, which then has to be reset afterwards.

Where my DAL methods are simple proxies to the stored procedure (i.e. they have the same signature and all they do is call a sproc and get the result with no other processing) I usually unit test the sprocs through the DAL.

If there is no DAL code that I can go through, or the DAL code does some additional processing or makes multiple calls, then I will often create some artificial DAL class in the unit test assembly. This will be a real bare bones thing that just gets the result back as a DataSet and so on. (I usually have the DAL create DTOs)

As for the resetting of data. I try an automate that by adding backups to the testing project. The unit test can then call a utility class that restores the backup on each run. However, this is a time consuming process even although it is automated and unit tests must be quick to run otherwise they won't get run - and having a number of backups needing restored during the unit tests doesn't help.

Unfortunately I don't have a good solution for that. In some instances it is possible to group a number of unit tests together and one require one backup to be restored. However, in this case the unit tests really have to be SELECT only tests because anything else would alter the state of the database. Sometimes it is possible to have a reset script that is quicker than a restore, but you have to be careful to ensure everything that is altered is reset - this can be a source of bugs in the unit tests if not done properly.

Interesting. However, I've been thinking more about the back end operation of the database. Specifically, how do you unit test DTS packages? With the ability to use CLR Integration in SQL Server 2005, I would imagine that you may eventually want to be able to perform unit tests in the database itself. Plus, you could use the power of the database to perform Pre/Post test conditions where the data was set up/rolled back to create a consistent view. Hmmm.

Whilst I've got very low length of experience in coding, about 9 months, a thought did occur to me. We've got a sql migration/refactor issue here.

Some tables will be split out, normalised etc.

Can one make stored procedures that produce specific query results then create a function that evaluates the output from one query and compare to another?

So oldtable1 query returns certain values for a given select criteria, perhaps one used already. This is compared directly to the values returned from what the developer thinks is an equivalent query in the new schema.

Then the differences reported?

Bit of a long shot, but that's roughly what I'm going to be doing with this database.

Can you recommend any resources for further reading?
I'm hoping for some test patterns. In particular I'm looking for those that address testing data-dependent code. I'm also looking for UI testing patterns.
Any ideas?

A few things that would aid in the write-up.
1) You show the automatic code-snippet but imply that it aids in testing. You may want to make it more explicit that writting the test is not a total loss, since you think about the interface you want to call, and then let VS 2005 create a stub for that interface. The stub generation is a useful tool with or without testing.
2) You should at some point mention the attirbutes [TestFixture] and [Test]. These seem to be the key to NUnit, which is only mentioned on passing.

You may want to make it more explicit that writting the test is not a total loss, since you think about the interface you want to call, and then let VS 2005 create a stub for that interface.

I don't understand your comment. None of the tests were a loss at any time. True, tests will initially fail. That is, after all, the point.

Crawfis wrote:

You should at some point mention the attirbutes [TestFixture] and [Test]. These seem to be the key to NUnit, which is only mentioned on passing.

That is beyond the scope of the article. The examples can be used with Visual Studio Team System's test framework also, as well as many others. The use of NUnit was only incidental. It was chosen because it appears to be the most commonly used unit test framework for .NET at the present time.

All this up front "keep it simple" work and refactoring, when a little up front "think about the problem" would have yielded a solution as well. And for more complicated problems, I think thinking is more important than launching into some simple/simplistic implementation. Where's the consideration for the overall architecture?

Sorry, but if anything, you're article, while well written, to the point, and of high quality, demonstrates to me that agile and XP methodologies are a very BAD way of going about the business of programming.

People are just notoriously impossible. --DavidCrowThere's NO excuse for not commenting your code. -- John Simmons / outlaw programmerPeople who say that they will refactor their code later to make it "good" don't understand refactoring, nor the art and craft of programming. -- Josh Smith

All this up front "keep it simple" work and refactoring, when a little up front "think about the problem" would have yielded a solution as well. And for more complicated problems, I think thinking is more important than launching into some simple/simplistic implementation. Where's the consideration for the overall architecture?

Well, although this question is outside the bounds of the article I'll answer it. The article was only a demonstration of how test first development should work rather than a discussion on the principles of the same.

The idea is that the overall architecture will be teased out as you refactor the code. You are right that just thinking about the problem would have yielded a solution too. There are a number of points in the article (especially the full version on my website) where I mention that the the solution should be obvious before even writing a test for it.

Marc Clifton wrote:

Sorry, but if anything, you're article, while well written, to the point, and of high quality, demonstrates to me that agile and XP methodologies are a very BAD way of going about the business of programming.

Well, that is a valid opinion to hold. While what I discuss is an ideal that I aspire to I have to admit that I don't always succeed. I have a tendency to see the solution in my head fairly quickly so I implement it, then test it to make sure it works.

On the other hand, I know of a number of people that do uses test first development (they call it test driven development) and say it works very well in the projects they work on.

The idea is that the overall architecture will be teased out as you refactor the code.

I feel that's a mis-application of refactoring. If refactoring means you're changing the model (inheritence, interfaces, design patterns, etc), then that's redesign, not refactoring. I've never having been one to buy into the refactoring concept to begin with, but I've come to understand that some things do fall under refactoring. They are, IMO, things like making a property public or private, extracting a method. Fairly benign things like that. But the architecture should be solid and should not be "teased out" by refactoring! I can't imagine working on a project where that would be the case.

A case in point--I'm working on a complicated video kiosk system for my client. There's all sorts logic involved in whether a movie should be played--things like whether the kiosk is open (schedule), time remaining, whether the door is open or not, whether the movie category matches what the person wants to watch, whether the movie is already being accessed by other kiosks.

I guess I call a spade a spade. The initial code was a prototype because I didn't know enough about all this DirectX stuff and the various other pieces to feel like I could come up with a good architecture. The initial version was a basically a redesign of the prototype. Recently I realized that a lot of this logic could be better handled (and more easily customized) by a decision table/tree. Do you call it "refactoring" when you rip out the guts of a system and replace it with something better, as I'm about to do? Do you call it "refactoring" when, in doing so, several classes and numerous methods are affected because the new design collects a lot of the logic in one place and unifies the analysis of the system state and actions? I sure don't. I call that redesign.

I realize I'm talking terminology, but I think it's important that the right terminology be used in the right place. Otherwise people misunderstand the issues involved when it comes time to do the work.

Colin Angus Mackay wrote:

On the other hand, I know of a number of people that do uses test first development (they call it test driven development) and say it works very well in the projects they work on.

I've taken the test first approach on a couple things, and while it's difficult to switch the work habits, I like it. But what I do differently, I guess, is that I put together the architecture first--stubs for the classes, the interfaces, the design patterns, then I write the tests, then I fill in the blanks. Otherwise, I'm just spinning my wheels, rewriting tests because the architecture keeps changing as I "refactor". And that's not productive.

People are just notoriously impossible. --DavidCrowThere's NO excuse for not commenting your code. -- John Simmons / outlaw programmerPeople who say that they will refactor their code later to make it "good" don't understand refactoring, nor the art and craft of programming. -- Josh Smith

I've taken the test first approach on a couple things, and while it's difficult to switch the work habits, I like it. But what I do differently, I guess, is that I put together the architecture first--stubs for the classes, the interfaces, the design patterns, then I write the tests, then I fill in the blanks. Otherwise, I'm just spinning my wheels, rewriting tests because the architecture keeps changing as I "refactor". And that's not productive.

That sounds like what I do. I'm quite good at thinking about the design that I've drawn some diagrams in Visio and written a whole bunch of stubs before I've got to writing any tests.

I know a few people that use TDD to tease out the design so it can work (or so I'm told). You've certainly got me thinking whether this is an ideal that works in as many situations as I'm led to believe. Perhaps where it works well is in situations like this article. There is only one public method, it has a well defined input, a well defined output and complex process to go from input to output. It is certainly the only time I've actually seen it work effectively.

I know a few people that use TDD to tease out the design so it can work (or so I'm told).

Actually, I've done that in one case. I had to put together an expression evaluator and I needed to test that the architecture was flexible enough to deal with additional functions not known at design time--a plug-in, in other words. I also needed to verify that the design handled notifications when the evaluation changed and also handled automatic re-evaluation when a dependency changed. And also, the ability to hold off re-evaluating the expression, in cases where, say, lots of dependencies changed all at once.

The result was a test framework that yielded several classes--the expression evaluator, the function interface for the plug-ins, and an autonomous class for managing dependency issues. It was quite helpful to iterate the tests and the design. It also showed that I needed separate unit tests for the dependency class and the expression evaluator.

And the really interesting thing was to develop the architecture and the architectural tests to support mock functions, an interesting issues because some of the real functions need to hit the DB and/or query the middletier. I wanted to be able to mock the entire data access layer and middletier communication so that the functions could be tested in a standalone environment.

The result was a very stable architecture, and one that was provably correct (as far as testing goes, hehe). The tests also provided a good template for other programmers to use for creating their own mock objects and of course plug-in functions. However, unit tests are not a replacement for *real* documentation, but it was a good start for providing examples.

So I can definitely see TDD used for design prototyping as well as the nuts and bolts of a specific class and its methods. That would be something to write a bit about.

I hope you don't mind all this feedback and my rather strong opinions. Personally, I wish I could really discipline myself to do more TDD, but it gets so darn complicated when dealing with failure modes of complex API's like movie/dvd players, USB devices, etc. I really haven't figured out how to do TDD right, so a lot of my feedback here is the result of my own muddling through the issues and not feeling very satisfied but knowing there *must* be some best practices and guidance buried in here somewhere.

People are just notoriously impossible. --DavidCrowThere's NO excuse for not commenting your code. -- John Simmons / outlaw programmerPeople who say that they will refactor their code later to make it "good" don't understand refactoring, nor the art and craft of programming. -- Josh Smith

I hope you don't mind all this feedback and my rather strong opinions.

No, of course not. It is good to have a calm and rational argument about something. Also, you've given me some things to think about - which is partly what I was after. It all adds to the learning process.

The reality is that I don't have a strong opinion for or against the ideas I showed in this article.

Over the last couple of years I've come to the conclusion that there is no one-size-fits-all methodology. Despite what others have told me or encouraged me to believe. I don't like the "this is the way we develop software regardless of the needs of the project". TDD is just one tool of many that can be used to write software. It also happens to be one idea that I felt had never really been explained very well. There has been a lot of abstract talk about it, but there have been few real examples of how it works. Hence this article.

I'd think the area of movie/dvd players would lend itself pretty well to unit testing. But you have to realize that most work goes into setting up the test data. Just have a disks with known content, and then you know what to expect to read. Vice versa for writing? Typically I'd set up some data with rectangular blocks, and some data with bad sectors etc.

TDD is not a substitute for good up-front design.
TDD can offer a couple of solid benefits however which can enhance or even compensate for incomplete or poor design. Additionally, it can provide developmental benefits that no amount of design can match. From what i've seen with our projects, the main perks for us include:

1. TDD reduces development risk by adding a degree of confidence and certainty that each tested component just *works*

2. TDD allows simpler refactoring by enforcing a contract on the output/operation of a given tested component; ie you've abstracted the implementation from the design meaning you can plug in any code that allows the test to pass. the major gain in this regard for us has been the ability to identify and refactor duplicated logic.

3. TDD assists developers in thinking about their design at multiple levels before diving right in. what am i trying to achieve? how does this fit in with other stories/tasks/functions?

4. TDD is mostly used as a means of upfront design at the iteration level; it assists us in fleshing out the detailed design as well as serving it's own purposes of testability and maintainability.

TDD is next to useless in the scenarios above if the coverage of your business function tests is not sufficient. Obviously if the tests aren't up to scratch (eg testing the wrong thing or false positives) you're wasting your time altogether.

It's just another tool in the kit, but TDD helps us in 90% of cases where we aren't given the chance to do complete up front design and where users change their minds...