Monday, May 20, 2013

Providing mocks through data providers

Over the weekend I played a bit with TestNG data providers for the
first time. This was in a scala program, but doesn't really have
anything to do with scala per se. I had something like this in my test
class. This is an example, it may not actually run, but it's
representative of what I'd written. Key here is that the mock objects
were being put together and set up in the data provider. The mocking
framework here is Mockito, but I imagine this type of problem could crop
up with other frameworks too.

This doesn't work, with a failure message saying that Track track2 does
not match Track track2. The debugger showed me the objects being
compared indeed had different identifiers, which led me to conclude that
the test was being run as two independent method executions, while the
data provider was executed only once. The mocks changed values on each
execution of the test method, while the data provider kept the mock
values from the first execution. The fix isn't entirely straightforward,
and required a combination of changes:

My tests involved many interacting objects, and several test cases to
check those interactions. I didn't like duplicating the mock setup code
from scratch for each test. However, reset on all mocks
(including the Position mocks) would cause loss of
behavior between test cases. So the creation of the Position
mocks had to move into the data provider. This version of the test ran
correctly:

Interestingly, Mockito developers seem to have been very concerned
about the potential
for reset to lead to poor coding practices, and had
to be convinced that there were legitimate uses for it. While my code
doesn't fall into the situations they had foreseen, calls to reset are
confined to the test setup.