[bey-buhl loo-tuh-fisk] : A distant relative of the Babel Fish. Recognisable by its glassy eyes and incomprehensible jabbering.

Wednesday, 22 July 2009

Factory Assembly Lines, Red Cars, and TDD

For the last five weeks I have been supporting developers on a project that makes full use of automated unit and integration tests, including the use of a mocking framework. The frameworks of choice are NUnit and Moq, which is fairly standard stuff. What makes our little project special, though, is that the developers were all (bar one) completely new to the concepts of unit testing and mocking! This has made for some interesting challenges. All the developers are highly capable and intelligent people, so they've adopted the new principles with amazing speed. But as anyone who's transitioned to TDD will know, the mindset required is very different from that of 'traditional' development. As such, I've had to do a lot of mentoring during the last month or so.

There's nothing quite like having a good example to go by when you're trying to learn something new. That's the case with TDD as well. I also think that in order to learn TDD and the test-first discipline you need something more. What's needed is a school of thought. Perhaps even a dogma. I think that this is far more important than specific examples of how to do specific things, because with the correct line of thinking you'll arrive at good solutions to any of your testing problems. It was with this in mind that I wrote what follows; a rather absurd testing scenario involving a factory assembly line and lots of red cars.

What I want to address now is the creation of tests, and the scope of this discussion includes what a test should test (the scope of the test, if you like), where boundaries should be drawn, and how such decisions should influence your design while you’re coding. For now, imagine that you are responsible for the production line of cars, and that you want to be able to test that a car is painted in the colour that you have specified.

How would you go about testing such a thing? The first thing that springs to mind is perhaps to instruct your assembly line (people and machines) to build you a red car, and then go have a look at the car when it’s built to see if it is, in fact, red. Such an approach would certainly work, but it would be a hell of an expensive test! What if the car came out the wrong colour? Or a different shade of red than you expected? You’d have to keep cranking out cars while making adjustments until you get the colour right, and that’s just not a sensible approach.

Of course, this example is absurd because you would never do that if you owned a car factory – but I’ve chosen this example exactly for that reason; you’re clearly doing too much work just to check that the car turns out with the right colour. Too much work. That’s a key thing to remember.

How could you improve on the process? Well, you might start by skipping the entire “build-me-a-car” bit and focus on the paint job itself. A car is painted by a big robot with lots of paint guns, and this whole mess takes place at the end when the car has been assembled and all non-metallic surfaces have been carefully masked etc. Since this is where paint is applied we can narrow our testing to this machine. We can put a piece of paper in front of the paint gun and ask it to colour the paper red. If things turn out unexpectedly, we can easily repeat the test. Paper is cheap, and even paint is much cheaper than a car. Not only that, but repeating the test is much faster also. This test is therefore infinitely better than the first test. We’re doing much less work, it’s costing less, and we’ve focused right down on the thing that makes a car red (or any other colour). Focus. That’s another key thing to remember.

Let’s stop and think about focus for a moment. In the first test, if a car turned out burgundy rather than the sports-car red you’d imagined, where would you make your adjustments before testing again? Sure, you may know that it’s the big ol’ paint robot at the end of the line that does the work, but how do you get your instructions to it? Is there a human involved? At what stage of the car assembly do you have to ‘input’ the colour? Right at the start? Does anything happen to your colour instruction while the car is being built? Does one human tell another, or is it written down on paper and later typed into a computer? It should be fairly easy to see that a test with such wide focus (or no focus at all) is prone to all kinds of environmental noise that can affect the output and thus make accurate testing difficult. So focus is important.

Now, we’ve narrowed our focus to the robot that paints the car, and we’ve devised a test that involves the robot spraying paint onto a sheet of paper. This is, in the scheme of things, probably a decent test. But we can do better. Does the paint gun produce the colour? No it doesn’t. The purpose of the paint gun is to deliver paint at the correct pressure and velocity, and to ensure that the nozzle produces a mist of paint with exactly the right droplet size etc. We can certainly use the paint gun to test colour, but since it’s got nothing to do with producing the colour of paint, we should see if we can create a better test. We should narrow our focus again.

If you keep repeating the process of narrowing your focus you’ll eventually end up at the part of the machine that mixes paint to create specific colours. Now the focus seems to be about right (you can probably narrow things down even further, though). The part of the machine that mixes the paint is what’s responsible for the colour that eventually ends up on the car. Arguably, you can take the whole paint element out of it because your paint is likely to be generic and colourless – so you can just focus on the mixing of pigments. But I don’t want to be too pedantic, either.

Now that we’ve got the focus right, we’ve got to get our test right. We’re trying to verify that when we ask for red, red is what we get. But if you think about it, red isn’t very specific either. Is it blood red? Burgundy? Maroon? A bit more on the pink side of the spectrum? To try and test for red isn’t specific enough. And here’s another thing to be mindful of. Be specific. Don’t test for red, but instead test for a specific shade of red. And so your test will start to change shape as well. Rather than testing for a colour you’ll find that you’ll be testing that the paint mixer dispenses the correct amounts of red, green, and blue (the basic components of the RGB colour system) in order to produce a specific shade of a specific colour. Now you’ve got a good test. A valuable test. A test you can write home about.

Such a test can be carried out without much work, it’s focused (it involves only part of a sub-system), and it’s very specific (it tests conditions for very specific input). All these things are good. In the context of the car factory you’ve now got a fast, cheap, repeatable, and accurate test that’ll verify that your factory is capable of cranking out cars in every colour of the spectrum.

Well, that’s actually a half truth. Or less than a half truth. You’ve only created a test for mixing pigments. You’ve not tested that the right amount of mixed pigment is added to the correct volume of paint. Nor have you tested that the paint gun actually works like it should and that the paint-robot moves like it should. And there isn’t a single test, yet, for any other aspect of the car assembly line. All this is beyond the scope of this discussion, but suffice to say that if you are a car producer hell-bent on making big bucks you’d apply the same principles as above to creating tests for every single aspect of your production line.

Now let’s try and relate this example back to software. The car assembly line would likely be implemented as a number of applications and/or services that each carry out specific tasks (basic assembly, welding, engine installation etc). The paint-robot might be one such application. The robot’s paint gun would be represented by a class, as would the pigment mixer. And the actual mixing of pigments would probably be represented as a single method on the pigment mixer class. That’s the unit you’d be testing. You wouldn’t write a test that invokes all those applications because it would require too much work and be too difficult, and it would result in an inherently inaccurate test.

So, when you’re writing tests and designing your code you should always make sure that what you’re working on lends itself to testing (it doesn’t require a lot of work), it is focused, and very specific. If what you are testing is part of something much, much larger then ensure that your unit is isolated. You do this by using mock objects for its dependencies. If you can’t use mock objects for whatever reason you should stop and think about why that is and see if you can change your design. Maybe you don’t have to. Maybe you can run your test in a slightly wider context without doing too much work or becoming too unfocused or non-specific. If you can, fine. If you can’t – change your design.

Careful thinking about how you’d test the code you are going to write will help you write better systems. As with anything, it requires practice and experience (and you’ll never stop learning) and it’s possible to go terribly wrong – but if every decision you make is the result of careful, informed consideration the likelihood of failure is minimal. Good tests are tests that are focused on one system, and specifically a single function of that one system – and that do very little work in order to test that function. Make the creation of such tests your goal, and you’ll quickly learn how to write clean, robust, and testable code.