Wednesday, February 06, 2008

Improvements to the unit test framework

In Factor, most unit tests compare the output of a word with a set of expected values:

[ 3 ] [ 42 36 - 2 / ] unit-test

The unit test fails if the returned values don't match, or if the quotation throws an error before returning.

However sometimes you want to ensure that a quotation does throws an error. The old way was to use unit-test-fails:

[ 3 "hi" + ] unit-test-fails

I renamed it to must-fail, which reads better:

[ 3 "hi" + ] must-fail

Sometimes, you want to assert that a specific type of error is thrown. The old idiom was to do the following:

[ t ] [ [ 3 "hi" + ] catch no-math-method? ] unit-test

However this is just boilerplate. There is a new must-fail-with word which encapsulates this idiom:

[ 3 "hi" + ] [ no-math-method? ] must-fail-with

Finally, because the overwhelming majority of the uses of the catch word were in unit tests, this word has now been removed. It has already most been phased out in favor of cleanup or recover, which are higher-level, and now that we have must-fail-with, there is no reason to keep catch around anymore.

The main problem with catch was that it would lead to ugly code; because it would either output the thrown error or f on success, it was almost always followed by a conditional.

The second problem was that it doesn't differentiate between no error and an error of f; while throwing f is bad style, it is not too far-fetched to imagine a scenario where buggy code throws f for some reason, and the error is subsequently ignored.

So, one less word, more boilerplate removed, it's business as usual in Factor-land.