Strong opinions, weakly held

As I have mentioned many times before, I’m a big adherent of unit testing. If you have a comprehensive suite of unit tests, seeing them all work before you commit a bug fix or feature to an application provides at least basic assurance that you haven’t screwed things up too badly. In the case of Web applications, the utopian goal is to be able to run your unit tests and feel assured that your application works without doing any testing through the browser at all.

There are many reasons why we’ll never see utopia, including client-side scripts and other problems related to display rather than basic input and output. However, it’s very difficult to consistently test an entire application going through it by hand using a browser, so you need to have a certain level of confidence in your unit tests. What I’ve found lately is that there’s a specific area where my unit tests usually fall short, and that’s in wiring forms up to actions in my application.

If you think of an application as an API, you can think of a URL as a method or function name, and its parameters as arguments. For example /search.cgi may take parameters like terms and language. When you write a unit test for this code, you use a test harness that pretends to post to that URL, and passes in the required parameters. If the call produces the output you expect, the test is successful. What my tests often lack, though, is a guarantee that the pages that submit to that URL use the correct parameters. There are three places where the parameters are referenced: the code itself (in search.cgi), the test, and the view that calls the URL. The view layer oftentimes goes untested.

So say someone wants to change the name of the terms parameter to query. They update the CGI program and the unit tests for it, and they update every page they can think of that calls search.cgi to update the parameter name. Most of the time, there are no unit tests that verify that the parameter names are correct in forms throughout the application, and even if you wanted to write such tests, there’s no good way to make sure that those parameter names are properly checked, since they live in the middle of freeform data. Your code coverage tool isn’t going to tell you that you missed a search form on some web page in the application.

Anyone know of a good approach for this problem? It’s been giving me fits lately.

It would then use assert_tag (or assert_select for those using 1.2+) to look for input fields with those names (“comment[name]”, etc.). It doesn’t really solve the duplication problem, but at least both would be specified in the functional test near to one another.

I work mostly in a J2EE Struts environment, and just jumped on the JUnit bandwagon in the past year. A real source of frustration for me has been testing DAO’s, since I place all the JUnit tests outside my JVM container, and we use JNDI lookups in an Oracle config file. I’ve managed to use MockStrutsTestCase with success to test the action forwards, but I have to say it seems like a rather trivial test.

I’ve been reading “JUnit Recipes”, by J.B. Bainsberger, and something he suggested really made sense to me. He says you really should not design your tests so that they test frameworks (like Spring) that have already been tested. That’s started me designing my classes so that the business logic is carried out in POJO’s, rather than classes that are heavily integrated with Spring/Struts or some other framework.

I realize this doesn’t answer your question directly — i.e. it doesn’t address Actions, except for the part about MockStrutsCase — but I think we’re dealing with the same subset of problems, i.e. how to test classes that aren’t pure POJO’s.