Friday, April 2, 2010

Two Sides

I was working on something last night and I was getting frustrated. Basically, I was writing a test of some code and having trouble distinguishing failure states. The code basically looked like this:

def method

begin

result = makeSOAPCall

rescue

result = "FAILED"

end

return result

end

In principle, this code isn't too shabby. It catches exceptions and handles them instead of blowing up, it returns a result that indicates a problem if there is one, etc. However, it has one major testability flaw: there's no way to see why you failed from a test. All you get to see is that it did fail or didn't.

My test was feeding values into the SOAP call, attempting to induce specific responses (successful and errors), and verifying that the parameters produced the results I expected. The frustration was that I couldn't see why something failed. So I was reduced to parsing through the logs where the actual error was recorded, and that was painful.

And then it occurred to me.

There are two sides to the equation. We - the collective team - owns both the code and the test code. We can make these reconcile. So I grabbed a developer and we changed the code so that the result object had an exception message on it which included the actual error. My test could simply handle that just like it handled seeing if the SOAP message succeeded or failed. Simpler test, and no loss of function in the product. Perfect.

It's easy to get caught up in needing to take the next step to get a piece of data out of a system you're testing. When it's hard, though, or brittle, it's worth asking if you can change the system to get the data out more easily. There are two sides to every contract; consider both of them.