Wednesday, September 19, 2012

In this release of ApprovalTests (available here ), we’ve only added one thing, but it’s a big thing. Specifically, it’s a big data thing.
My thanks go out to John Heintz for helping me to create this.
Here's a video of the whole thing:

Unit Testing HadoopMapReduce

Before we go into the details of everything involved in HadoopMapReduce, let’s go through an overview of how we’ve been testing our MapReduce jobs. In this example, we are doing this common demonstration of a word count MapReduce job. The idea is that sentences will come in, they will be split into words, count them up, and summarize in our reduce job.

Visualizing the System

Unit Testing provides many different advantages to a programmer. And all of them have different importance to different people, but one of these advantages is specification. In particular, with MapReduce, being able to understand the flow and transformation of the data can be more enlightening than the other aspects of Unit Testing.
One of the things that we strive to do with ApprovalTests, is create output that is meaningful and insightful. To demonstrate this, I’m going to start output of my word count unit test.

As you can see, this output is a nice way of visually understanding what comes into the job, how the WordCountMapper transforms it, and what the final result is. Let’s take a look at the code that produced this result, and then we’ll break it down into individual segments.
Here is the entire line to create that output:

MR Unit

HadoopApprovals sits on top of MR Unit . You will need to grab it and include the jars. The javaDocs for HadoopApprovals mention all of the jars needed to run it. They are:

hadoop-client-1.0.3.jar

hadoop-core-1.0.3.jar

hadoop-test-1.0.3.jar

log4j-1.2.15.jar

mockito-all-1.8.5.jar

mrunit-0.9.0-incubating-hadoop1.jar

HadoopApprovals

HadoopApprovals has three main functions: the ability to test the mapper, the reducer, and the map reduce combination. To make this easier, we are going to use extra information with the generics called SmartMappers and SmartReducers. We will talk about those in the next section.

Testing a Mapper

Here’s the method to test the mapper:

verifyMapping(SmartMapper mapper, Object key, Object input)

To test a mapper, you need the mapper you’re going to test and the keyValue pair going in. Many times, mappers do not actually use the key that comes in, but regardless, you will need one anyway. Do not make it null.

Testing a Reducer

Here’s the method to test the reducer:

verifyReducer(SmartReducer reducer, Object key, Object... values)

When testing a reducer, you will need to pass in the key plus a set of values for the key. You will notice you can pass in regular strings and regular integers instead of Text and LongWritable. Again, this is because of the SmartMappers we will talk about in the next section.

Testing a full MapReduce job

As you might expect, this is almost identical to testing a mapper. Except with the extra addition of a reducer.

SmartMappers

I wrote a blog about getting generic type information at runtime from Java. You can read it here:

The long a short of it is, if you add an extra interface to your mapping and reducing jobs, you can avoid a lot of boiler-plate code needed to state what the runtime types of the input and output are. You can do this by simply having your MapReduce jobs extend SmartMapper instead of Mapper.

“What? I don’t want to change my stuff!”

Fair point. If you want to test against an existing mapper or reducer, and you don’t want to change it to use the SmartMapper extensions, you can always wrap it using the MapperWrapper (bonus: fun to say out loud) or ReducerWrapper. Here’s a sample of how to do that:

Event approvals

Similar to the example above, another aspect of code which is often hidden (or implicit) is which events are wired-up to your form. This can be particularly troublesome when changing an existing, unfamiliar piece of code. For example, it it easy to accidentally remove a button-click event.

Event approvals allows you to easily lock down existing events which are associated with an object or a form. To do this, simply call the following:

EventApprovals.VerifyEvents(myObject);

This new method will only allow you to verify events which are directly associated to the object which is passed in. Unfortunately, many times you don’t want to say ‘What are the events associated with this button"? rather you actually want to say ‘What are the events associated to this form, even though the button is on the form and the event is associated to the button (rather than the form)?’ This is now no problem, since we took this scenario into consideration and also created a convenience method for the latter scenario.

To test all the events on a form and its immediate children, simply call code as shown below:

WinFormsApprovals.VerifyEventsFor(new DemoForm());

WPF support for Controls

It’s always been possible to test WPF windows with a simple:

WpfApprovals.Verify(window);

This would render the window to a .png image and verify against .approved file.

But sometimes you want to scope smaller to an individual control. Unfortunately, we had previously neglected this scenario – no longer! You can now test an individual WPF control with the same call as shown below:

Entity Framework support

People have long pondered how to test Entity Framework. We have talked about this before, but previously testing EF with ApprovalTests had always required a large number of steps. Now you can test EF by simply calling

This method will implement a ‘test-the-weather’ scenario which will verify the resulting SQL without the need to be able to connect to a database. However, if this method fails, then it will connect to referenced database and will execute the generated query and will return extra information about WHY the test failed. I plan to create more detailed tutorials on the inner workings of this new method soon.

Let’s give you one more look at this method signature.

publicstaticvoid Verify<T>(ObjectContext db, IQueryable<T> queryable)

Email support for attachments

Previously, we added the method

EmailApprovals.Verify(message);

Unfortunately, when the email message contained attachments, then the GUIDs that those attachments created caused issues with test output because they were not consistent over multiple test runs. This bug has been fixed in this release. To learn more about approving email messages, watch this video.

Testing Email with ApprovalTests

FileLauncher with DelayReporter

There has been a great deal of demand to use Cassini-dev after Jim Counts demo’d it during his ASPConf video on testing ASP.MVC views. Unfortunately, when using this method it’s a bit tricky to use a file launcher, since the web server dies BEFORE the browser can hit it. We came up with a simple solution which adds a tiny delay into the method execution so that this is no longer an issue.

[UseReporter(typeof(FileLauncherWithDelayReporter))]

Well, that’s all for this release. As always, if you have any questions about ApprovalTests, tweet them with the hashtag #ApprovalTests, I monitor that and will answer you promptly.

It was great to see this level of professionalism in a demo session; an area usually reserved for cowboy programming of hello world demos. Don’t get me wrong, Jim still does a simple “Hello World” type demo, but he applies all the rigor and professionalism I would want in a real project to that example.

Saturday, May 19, 2012

For many years, the views (the html returned from an rendered MVC controller) were an extremely difficult thing to test. Like many unreachable things, they were declared “not worth it” and ignored.

Until now,
with ApprovalTests we have created a “simple” way to test the rendered output of a view. I say “simple” because there is a fair amount of overhead for the 1st test, which means we wanted to give you some resources to get it working for yourself.

I created a short video tutorial, and Jim Counts put together a fabulous 3 part blog + Sample Code.

Now you can decide for yourself how important it is to test the views.

Code:

Saturday, May 5, 2012

This is the pattern I find myself in day after day while practicing TDD or BDD.

Step 1?

Usually I start at the whiteboard. It is the easiest place to start as I sketch out the scenario I want to program, but not always. Sometimes a mockup file (image, html, or xml) is provided and I start at the result, sometimes I’m give a scenario already in English, and sometimes I have to fiddle with the code first to see what’s happening.

No Beginning, No Ending

The great thing about the circle is no matter where a start, the flow should always be there in the end. If it isn’t, I take this as a smell that I am neglecting something. Also, while I wrote the arrows going clockwise (the normal way I view this process) they can equally go in reverse.

Smell 1: No Result –> Whiteboard

If the result does trigger the original image on the whiteboard (usually long erased by the time I revisit the code), then this will be a maintenance problem when I try to remember why the code looks the way it does when it changes.

Smell 2: No Code –> English

If the resulting code doesn’t make the comments so easy that they should be deleted for not adding any value, then my intention (so clear when I’m writing the original code) will not be there when my fellow programmers, or myself even a few days later, what the understand the code later.

Friday, April 27, 2012

Email Verification

Almost every site sends some sort of email. These have been hard to test, usually requiring some sort of email to be actually sent. Now you can simply say:

EmailApprovals.Verify(new MailMessage())

This is particularly nice with a

[UseReporter(typeof(FileLauncherReporter))]

As it will open in your desktop email client (like outlook, not gmail) making it very easy to see the results. If you’ve given up on desktop clients, you can grab one from www.ninite.com
I made a video about it here:

Verifying Email

Better Rdlc Support

Previously, if you wanted to approve a Rdlc report with multiple data sources you had to use Tuples, which got a bit annoying. In this release I am taking advantage of the anonymous enumerator syntax. The result is

Tuesday, April 10, 2012

FrontLoadedReporter Attribute (For Build Systems)

You can now add the following code to your assembly.

[assembly: FrontLoadedReporter(typeof(NCrunchReporter))]

This is only allowed at the assembly level, and must be a IEnvironmentAwareReporter. If the reporter is allowed in the current environment, it will circumvent all other reporters. This is very useful for Build Systems where you want to override the behavior of the reporters.

IEnvironmentAwareReporter is file specific (better composition)

The IEnvironmentAwareReporter interface has changed from

bool IsWorkingInThisEnvironment();

to

bool IsWorkingInThisEnvironment(string forFile);

this allows for better composition of reporters. For example I can now combine DiffReporters that only do text with DiffReporters that do images. If you have an Image it will fall thru to the ImageReporter even if the TextDiff Program is on your system.

This is especially nice for…

CodeCompareReporter

My new favorite text diff tool is Code Compare, mainly because it integrates directly into Visual Studio 2010. You can download it here (it’s free)

Dictionary Support

This will now create the expected approval result. I don’t know how this didn’t exist from the very start. It is worth noting that keys will be presented in their natural ordering (to allow consistency.)

EF and Mvc3

I added a Mock Saver, LambdaLoader & SingleLambdaLoader to the Enitity Framework support.
I also added .Explict to the Mvc3 Toolkit. I will make videos about this approach soon.

Wednesday, March 21, 2012

I recently did 3 videos in my ‘Using ApprovalTests in .Net’ series that deal with how to test difficult code. These video actually have very little to do with either .Net or ApprovalTests, but they are common issues you run into when unit testing.

Seams

Testing difficult Code comes down to 2 things:

Can I call the Method?

Will the Method run to completion?

And there are 2 techniques I use to deal with these, that individually are useful, but together are amazing.

1. The Peel

The hardest part about long methods is you have to run all of it to test even a small part of it. The idea behind the peel is to break the method up into easy to run pieces. Here’s the video:

The Peel

2. The Slice (Mocks)

Even if you get into the piece you are looking to test, you might not be able to get all the way through it. That’s where mocks come into place. By slicing off the connection to the actual trouble spot, you can easily fake it, and return to easy to run code.

The Slice

3. The Peel & Slice

So now that you have these Mocks, how do you get them to be used? While there are many forms of dependency injection (inheritance, factory, DI frameworks, IoC) the easiest of all is “Parameter passing”. Of course, to do that you need a nice seam that passes in the default dependency. Sound familiar? This is why the Peel & Slice work so nicely together.