Wednesday, May 28, 2014

Why I don't mock

Well, it's impolite, isn't it? But seriously,
when I first heard about mock object testing, I was excited, because it
certainly sounded like The Right Thing™: message-based, checking
relationships instead of state, and the new hip thing.

However, when I looked at actual examples, they looked sophisticated
and obscure, the opposite of what I feel unit tests should be:
obvious and simple, simplistic to the point of stupidity. I couldn't figure
out at a glance what the expected behavior was, what was being
tested and what was environment.

So I never used mocks in practice, meaning my opinions could not
go beyond being superficial. Fortunately, I was given the
task of porting a fairly large Objective-C project to OS X
(yes, you read that right: "to OS X" ), and it was heavily
mock-tested.

As far as I could tell, most of the vague premonitions I had
about mock testing were borne out in that project: obscure
mock tests, mock tests that didn't actually test anything except their
own expectations and mock tests that were deeply coupled to
implementation details.

Again, though, that could just be my misunderstandings, certainly
people for whom I have a great deal of respect advocate for
mock tests, but I was heartened when I heard in the recent
DHH/Fowler/Beck TDD death-matchesfriendlyconversations that neither Kent nor Martin are
great fans of mocking, and certainly not of deeply nested mocks.

However, it was DHH's comments that finally made me realize that
what really bothered was something more subtle, and
much more pervasive. The talk is about "mocking the database",
or mocking some other component. While not proof positive, this
kind of mocking seems indicative of not letting the tests drive
the design towards simplicity, because the design is already
set in stone.

As a result, you're going to have constant pain, because the
tests will continuously try to drive you towards simplifying
your design, which you resist by putting in mocks.

Instead of putting in mocks of presumed components, let the
tests tell you what counterparts they want. Then build those
counterparts, again in simplest way possible. You will likely
discover that a lot of your assumptions about the required
environment for your application turn out not to be true.

For example, when building SportStats v2 at the BBC we thought
we needed a database for persistence. But we didn't build it
in until we needed it, and we didn't mock it out either. We
waited until the code told us that we now needed a database.

It never did.

So we discovered that our problem was simpler than we had
originally thought, and therefore our architecture could
be as well. Mocking eliminates that feedback.

So don't mock. Because it's impolite to not listen to what
your code is trying to tell you.

I don't understand the database example. If I design outside-in with mocks, then I will either reach the point where I'll need some collaborator that will provide me some information or not. In the former case, I mock that role without thinking about the persistence mechanism and other implementation details. At this point, I'm still not commited to use a database.

Mocking is useful in cases such as testing a module that depends on a third party API. You mock the API so that you can test only the module itself, and not the external API it depends on. You can test the external API itself separately. This way if your module tests fail, you know it is a problem with the module itself and not with the external API dependency.

> Sounds very similar to Uncle Bob's story when they were creating FitNesse.

They end up using a file system store instead of a RDBS, if I recall correctly. At first they created a stub, later some in-memory storage. This seems to be entirely orthogonal to mocking to me. In fact they used stubs too.

this is something I realized as well. but it is very hard to explain, and I actually never managed to convey what this means completely. in this article I can see the same hardship in exposing this concept.

I have this mental model of the code: there is an evident input and output (parameter and return values) then there is an external state (database data, third party api call) and an internal state (local variables and members)

input and output are dead easy to test if there are no operation done on external and internal state.

operation on internal state are as quell easy to prepare and verify exposing the right method on the object to be tested.

that leaves us with the external state. this is where mocking actually helps but: I prefer decoupling instead of mocking. I don't query the database, I create a dao that is passed as local state - then the dao can be made to return fixed data on testing. easy to see, easy to follow, easy to debug.

now, you have to test the dao, and here you are stuck with moking or integration test, however, this pushing to delegation of external state make the logic code dead easy to test.

as external state routines take exclusively the form of load and store, even if they are painful to test, they are easy as well.

If the lib didn't fail testing but would have caused some other component to fail testing surely you would want to know about that even if it is harder to track down the bug than if it had been found in the tests for the library.

> operation on internal state are as quell easy to prepare and verify exposing the right method on the object to be tested.

Please don't. This is almost always wrong when unit testing in an OO language. You should not expose internal state in order to test it. This would degrade your object's public interface and would make your design worse. This is actually one case where test doubles work well.

Instead of putting in mocks of presumed components, let the tests tell you what counterparts they want. Then build those counterparts, again in simplest way possible.

Consider that you need to integrate with third party systems. In fact, the name of your application is about integration with a proprietary service, software or platform... Sometimes not even proprietary, but the one that cannot be run on your machine.

Do you build Google AdWords, because you need to integrate with it? Or do you mock it out? Do you try to reverse engineer SAS? Rewrite Hadoop? Rewrite SAP? and the list goes on... There are thousands of systems that you can't just go and use the actual object/API/whatever.

I don't like mocking as well. I don't advocate or do it for anything other than external(uncontrollable) component. But then again... I don't do excessive unit testing, preferring integration testing.

It is also a bad habit of writing unit tests before you start coding. As those will either make you adhere to a certain design/architecture(that is fixed beforehand) or be a nightmare to maintain. Mocking components is just the extension of that issue.

@Anoymous: Yes, I was quite amused when I saw Uncle Bob's Fitnesse story when I saw his Ruby Midwest 2011 keynote: Architecture, the Lost Years. While I don't agree with everything he says, there is a lot of good stuff there.

And no, they did not stub out (which is different from mocking) the storage. They just didn't need storage at all for al long time. When they started needing storage, they also didn't stub it out, they just implemented the simplest thing that could possibly work, which happened to be a file store.

So they discovered a lot about their problem space, particularly that it was simpler than they had suspected, just as we did. Simplicity is good, and very often the drivers of unnecessary complexity are we the programmers, mostly due to faulty assumptions.

I think more people would make this discovery if they let their tests guide them in their architectural discovery. Mind you, the tests do not induce an architecture, and certainly not a good one. You do need to design it yourself, and it is almost always helpful to think about that architecture beforehand. You just want to adapt that design to the discoveries that you make along the way, and tests are a good mechanism for enhancing the feedback you get.

Mocks in many cases let you shut out that feedback making it easier to not discover anything interesting about your architecture.

One reason to mock objects is so you don't have to setup supporting data needed for a test case, you just embed the data in the mock objects. Many call those stubs instead of mocks, which I find make for more understandable tests. For example, testing a credit card processing back charge would first require you to run something to create the charge in the first place.

Mocking is pretty much a must if you do unit testing. You can only have coarse grained tests without it (assembling most / all of your app for your tests).

Eg. all your tests may blow up because of a small issue in a commonly used dependency somewhere down the chain. You may get away with that on a small/simple project, but can be very time consuming on a more complex one.

When it comes to integrations / infrastructure you need to do some stubbing in any case - unless you go even further and rely on integration tests only, which further limits the set of projects that can allow to do this.

Hi Marcel. I am wondering what you think of the following paper where Steve Freeman, Nat Pryce, et al. see mocking as a tool for interface discovery.

http://www.jmock.org/oopsla2004.pdf

Perhaps you can shed some insight on their mocked presumptions in the context of roles and expected interactions.

Another thing I'm curious about is DHH's comment on "mocking the database". DHH assumes that he needs to mock a database, but maybe what he really needed to mock was a component playing the role of a data store, which could be a file store, data base, web service, etc. After mocking the role of a data store, we could implement it using a file store to start.

As someone trying to grok both schools of TDD, I'm very curious to hear your thoughts!