Visual Studio Fakes Part 2 - Shims

Wednesday, April 25, 2012 @ 09:00 AM

Let me start by saying Shims are evil. But they are evil by design. They
let you do things you otherwise couldn’t do, which is very powerful. They let
you do things you might not want to do, and might know you shouldn’t do, but
because of the real world of software, you have to do.

The Catch-22 of Refactoring to Enable Unit Testing

In the first part of my series on VS11 Fakes, I reviewed Stubs, which are
a simple way of creating concrete implementations of interfaces and abstract
classes for use in unit tests.

But sometimes it happens that you have to test a method where the dependencies
can’t be simply injected via an interface. It might be that your code depends
on an external system like SharePoint, or simply that the code news up or uses
a concrete object inside the method, where you can’t easily replace it.

The unit testing agilista have always said, “Refactor your code to make it more
testable,” but therein lies the rub. I will again refer to the esteemed Martin
Fowler for a quote:

Refactoring is a disciplined technique for restructuring an existing body ofcode, altering its internal structure without changing its external behavior.

How do you know if you are changing its external behavior? “Simple!” says the
Agilista, “You know you didn’t change it as long as your unit tests still
pass.” But wait… we don’t have unit tests yet for this code (that’s what
we’re trying to fix), so I’m stuck… Catch-22.

There was only one catch and that was Catch-22, which specified that a concernfor one's safety in the face of dangers that were real and immediate was theprocess of a rational mind. Orr was crazy and could be grounded. All he had todo was ask; and as soon as he did, he would no longer be crazy and would haveto fly more missions. Orr would be crazy to fly more missions and sane if hedidn't, but if he were sane he had to fly them. If he flew them he was crazyand didn't have to; but if he didn't want to he was sane and had to. Yossarianwas moved very deeply by the absolute simplicity of this clause of Catch-22 andlet out a respectful whistle.

Joseph HellerCatch-22 Chapter 5

He’s crazy if he wants to go fight, but if he says he really didn’t want to
fight then he isn’t crazy and so needs to go fight. We have the same problem.
We can’t test because it needs refactoring, but we can’t refactor because we
don’t have tests.

Shim Your Way Out of the Paradox

Example 1 -

To see what this really looks like, let’s look at some code (continued from the
example in Part 1).

"Untestable" System Under Test Code

123456789101112131415161718192021222324252627282930

namespaceShimsDemo.SystemUnderTest{publicclassCustomerViewModel:ViewModelBase{privateCustomercustomer;privatereadonlyICustomerRepositoryrepository;publicCustomerViewModel(Customercustomer,ICustomerRepositoryrepository){this.customer=customer;this.repository=repository;}publicstringName{get{returncustomer.Name;}set{customer.Name=value;RaisePropertyChanged("Name");}}publicvoidSave(){customer.LastUpdated=DateTime.Now;// HOW DO WE TEST THIS?customer=repository.SaveOrUpdate(customer);}}}

There is one minor change from our previous example. We are now setting the
LastUpdated property on the Customer object before passing it to the
repository. (I know this might not be the best way to do this, but go with
me…)

How do we test that Save sets the correct value to LastUpdated?

You might start by writing a test like this. This uses the same Stubs techniques
I showed in the last article and tries hard to deal with the variable nature of
the LastUpdated property.

A Not-so-good Test for LastUpdated

123456789101112131415161718192021222324252627282930313233

publicclassCustomerViewModelTests{ [Fact]publicvoidSaveShouldSetTheCorrectLastUpdatedDate_WithoutShim(){// ArrangevarsavedCustomer=default(Customer);// nullvarrepository=newStubICustomerRepository{SaveOrUpdateCustomer=customer=>savedCustomer=customer};varactualCustomer=newCustomer{Id=1,Name="Sample Customer",LastUpdated=DateTime.MinValue};varviewModel=newCustomerViewModel(actualCustomer,repository);// Actvarnow=DateTime.Now;viewModel.Save();// AssertAssert.NotNull(savedCustomer);// We will use a 10ms "window" to confirm that the date is "close enough"// to what we expect. Not ideal, but it should work... most of the time.vardelta=Math.Abs((savedCustomer.LastUpdated-now).TotalMilliseconds);constintaccuracy=10;// millisecondsAssert.True(delta<=accuracy,"LastUpdated was not appx equal to expected");}}

There are, of course, a few issues with this code, most notably the bit below
the comment explaining the 10ms window. How small can we make the accuracy
variable before the test starts to fail? How close is close enough? What about
when the viewModel.Save() code does extra work eating up some of that time?

A better way to test this is to use a Shim to override System.DateTime, causing
it to always return a predictable value. That test would look something like this:

On line 21 we use Shims to override the Now static property getter on the
DateTime object. This replacement is for all calls to DateTime.Now in the
AppDomain, which is why we are required to provide a scoping region for our
replacements with the ShimsContext.Create() block.

But now this test is no longer “flaky”. It has no dependency on your system
clock, on the length of time that Save() or any other code takes to run. We
have controlled the environment and isolated this test from the external
dependency on DateTime.Now.

This example shows one issue that can make code difficult to test, but of
course there are more.

In this example, we have a method that is creating an instance of the
WPF DispatchTimer class. It uses the timer to track when 30 seconds
have elapsed and then it refreshes a property on the view model (presumably
this collection is bound in the XAML to a list on-screen) by calling
RefreshData().

How can we test this code? There are a few things we might like to test,
including:

In this example we use Shims ability to detour all future instances of an
object by using the special AllInstances property on the generated Shim
object. Since these methods are instance methods, our delegate has to include a
parameter for the “this” pointer. I like to use @this for that parameter to
remind me what it is, but I’ve seen other people use that or instance if
you dislike the idea of using a C# keyword in your code.

We override the Interval property setter (which takes a timespan, hence the
name IntervalSetTimeSpan) and the Start method. By stashing away values into C#
closures, we can call sut.StartTimer() and then verify that what we expected
did in fact happen.

Example 3 - Using Shims to wrap an existing instance

One more example will wrap up my introduction to Shims.

We will now take a look at how we might implement the second test in my
list above. We want to confirm that the RefreshData method is called
only after 30 ticks have elapsed.

Let’s take a look at how we would test this using Shims. In this case we
will still need to control the DispatchTimer, but we will also need to
override the implementation of RefreshData(), which is a method on the
class we’re testing.

Shimming a specific object instance

123456789101112131415161718192021222324252627282930313233343536

[Fact]publicvoidRefreshTimerCallsRefreshAfter30Ticks(){using(ShimsContext.Create()){// Arrangevarcustomer=newCustomer();varrepository=newStubICustomerRepository();varsut=newCustomerViewModel(customer,repository);varrefreshDataWasCalled=false;newShimCustomerViewModel(sut){RefreshData=()=>refreshDataWasCalled=true,};// Setup our override of the Dispatch timer// We will store the tick handler so we can "pump" it// and noop override Interval and Start just to be safevartick=default(EventHandler);ShimDispatcherTimer.AllInstances.TickAddEventHandler=(@this,h)=>tick=h;ShimDispatcherTimer.AllInstances.IntervalSetTimeSpan=(@this,ts)=>{};ShimDispatcherTimer.AllInstances.Start=(@this)=>{};// Actsut.StartTimer();// AssertrefreshDataWasCalled=false;// Clear out the any calls that happened before the tick()for(vari=0;i<29;i++)tick(this,null);// Tick 29 timesAssert.False(refreshDataWasCalled);tick(this,null);// Tick one more timeAssert.True(refreshDataWasCalled);}}

While nobody would call this a pretty test, it does what we asked it to do, and
has let us test one aspect of this method we otherwise wouldn’t have been able
to test.

Resolving the Catch-22

The Catch-22 that I mentioned at the start of this article can be restated like this:

You can’t test the code because it has internal dependencies that make it untestable

You need to refactor it to fix the dependency

Since you don’t have any tests for this code, you aren’t safe to refactor

See #1

Hopefully you can see that Shims help you get out of this dilemma. With Shims we can
rework that list to something like this:

You can’t test the code using traditional techniques because it has dependencies
that make it untestable.

Before you can refactor it, you need to create a test that asserts its current
behavior.

You can use Shims to create this characterization test, enabling you to refactor
safely.

Once you have it under test, you can safely refactor and make the dependency
explicit and replaceable using traditional (e.g. interfaces and stubs) methods.

The characterization test should continue to pass after performing the refactoring.

Once the refactoring is complete, you can create new tests that do not use Shims
(but may use Stubs) to test the code.

Then you can remove the Shims test.

It is a few more steps, but it lets you have a clear, intentional way to refactor
code that otherwise would have required “refactoring in the dark”. The Catch-22 is
resolved and you can continue to improve the design and testability of your code.

I do recommend people go all the way with this approach and refactor the code
until the Shim-based test is no longer required. Every time you need to use
something like Shims to test something, it is telling you that you have a
design problem. You probably have high-coupling and low cohesion in that
method.

Refactoring is the act of intentional design, and you should always take the
opportunity to make your design better. Shims can be used to get out of this
impasse, but if you don’t think about the problem you will end up with a Big
Ball of Mud for your design.

Some final words about Shims

As the famous literary quote goes (Voltaire, Socrates and even Spider Man’s Uncle Ben):

With great power comes great responsibility.

The examples I’ve shown in this post are all examples of design flaws around
coupling and cohesion. Knowing that, we should always feel a bit “dirty” about
having to use Shims to test something. Every time we do it, we should remember
to go the extra mile afterwards and refactor it away if we can (my steps 4-7
above).

The Catch-22 of untestable code is real. Getting out of it is hard. Shims are
designed to help you get out of this trap.

Shims may be evil from a purist design and TDD sense, but in the real world we
are often faced with code we a) don’t control or b) didn’t write and which
doesn’t have any tests. Use Shims to get out of that, but always continue on
and fix your design issues.

Conclusion

Visual Studio 11 includes the new Fakes library for creating isolated unit
tests. It includes two kinds of Fakes:

Stubs for creating lightweight, fast running concrete classes for
interfaces and abstract classes your system uses. I reviewed Stubs in my
last article.

Shims for intercepting and detouring almost any .NET call. Shims are
particularly useful for removing internal dependencies of methods, and for
getting you out of the “testability Catch-22”.

Hopefully you have found these two articles useful. For more information on Fakes
please take some time looking through the MSDN Documentation.

Comments

Peter Provost is a life-long-learner, hacker, maker, agilista,
musician and heavy metal fan. He lives in Colorado with his wife
and two children and works as a Program Manager Lead on Microsoft's
Visual Studio ALM tools.