Menu

Thoughts and notes on unit testing

I’m not a fan of unit testing. As a matter of fact, I have never written any unit tests in my career. Well, I almost did for one of my previous projects that was growing huge, but the client was more interested in seeing the functional requirements met over the non-functional ones (probably because they don’t see progress in it). The project manager could’ve convinced the client that the project needed unit tests, but that’s another topic.

So now I’m facing a project that’s growing beyond what I expected, primarily because I wanted to add features only as the need arises. And as the project grows, I’m encountering circumstances where I already need to refactor a lot, which risks breaking the code. I can test the whole project everytime I refactor the source but I’m too lazy to test everything over and over so I decided to add unit tests to the project.

In search for a unit testing framework that’s applicable for the project, I found out about GAEUnit.

GAEUnit is a unit test framework that helps to automate testing of your Google App Engine application. With a single configuration change (it can be completed within 30 seconds), your unit tests can be run in the real GAE app server environment using a web browser.

I was able to setup GAEUnit after getting over a bit of misconfiguration (the handler for gaeunit.py was supposed to be placed on top of the other handlers in app.yaml and I don’t know why) and used the modified ‘webtest’ module that came with the zipped archive.

So I was set to write my unit tests. Then I realized that I didn’t know which parts of my code needed urgent unit testing except for one, which is to check whether the mailer is working without actually sending an email. Which pieces of my code needed unit tests? I also didn’t want to write too many unit tests. I started to wonder how much unit tests other developers are including in their projects. How do I write tests for functions that needed parameters that are valid and should come from the database? Should I query for valid values within unit tests? Should I write unit tests for those queries that are only used for the unit tests? Should I write unit tests for my unit tests? Or perhaps I can write a program that automatically writes the unit tests that I need?

I found several good points from an article by Tim Burns, Effective Unit Testing, which answers most of my questions:

The crucial issue in constructing a unit test is scope. If the scope is too narrow, then the tests will be trivial and the objects might pass the tests, but there will be no design of their interactions.

If the scope is too broad, then there is a high chance that not every component of the new code will get tested. The programmer is then reduced to testing-by-poking-around, which is not an effective test strategy.

Start with a feature that is described in the requirements document and work down until a method doesn’t need a unit test. That way, tests are grouped according to major feature and should include as many unit tests that deal with the feature in question as are necessary.

If the code is simple enough that the developer can just look at it and verify its correctness then it is simple enough to not require a unit test. The developer should know when this is the case.

If error handling is performed in a method, then that method can break. Generally, any method that can break is a good candidate for having a unit test, because it may break at some time, and then the unit test will be there to help you fix it.

The programmer should know that their unit testing is complete when the unit tests cover at the very least the functional requirements of all the code. The careful programmer will know that their unit testing is complete when they have verified that their unit tests cover every cluster of objects that form their application.

I’m still looking for good articles on unit testing. Feel free to post your tips and resources.

Post navigation

5 thoughts on “Thoughts and notes on unit testing”

I suppose the main thing about tests is that they work as scaffolding. They don’t guarantee that everything works. They just give you that extra bit of confidence you need when refactoring. It’s easier not to screw something up when you have plainly stated the constraints and corner cases code has to abide to. 🙂

Of course tests are not without their problems. If not maintained, you might end up with a mess. Now you have ~two~ issues instead of one. It’s not enough just to write tests. It’s important to keep them up to date and legible.

Sometimes I prefer to use testing to drive my development work. This is particularly handy if I’m familiar with the tool I’m using and I’m just exploring some new domain constraints of which I know. Just recently I wrote a tiny tokenizer this way. I sketched out expected results and wrote the code to comply to my definition. As you might well know this approach is known as Test Driven Development (TDD).

Besides TDD there is a whole family of xDD approaches. These include behavior, documentation, domain, acceptance testing, even bug (ok, this is a joke!) etc. drivers. The key idea here is that you derive the constraints for your code somehow.

The approaches vary based on how they state their starting point. You might want to use something that the customer himself may be able to come up with (stories). Then it’s up to you to map it to something more tangible. You could end up using a tracer bullet approach (Hunt, Thomas) to sketch out the whole app and just fill in the details as you go.

There are many approaches to development and testing altogether. It’s up to you to find the ways that work well in your context. Good luck! 🙂

I always found unit tests to have more value in developing then in actually testing. To be able to write tests in first place often requires keep code more loosely coupled, and to abstract IO functions more. It did not help at all in logic bugs or memory leaks. There is still great need for good systems and regression tests.