Fixture Based Fluent Unit Testing

Dec 9, 2018
Alex Haslehurst

Given my last few posts, you’d be forgiven for thinking that this blog has an underlying testing theme. As unintended as it may be, since I’ve written yet another lightweight unit testing framework that works significantly different to my previous specification structured approach, I think it qualifies for another post. given-fixture is intended to support writing really concise but readable tests built from many small, fluent extension methods that configure a fixture for taking care of all that nasty boiler plate. Me and the rest of my team have written hundreds of unit tests with this. We seem to be having great success and continue to refine it regularly. So please take a look, it’s available on GitHub and NuGet.

Why?

What’s up with specification structured tests?

I’ve been writing specification structured unit tests in C# for a few years now and other than a few occasions, mainly when testing over ambitious services who’s behaviour tree had grown totally out of control, I’ve had a great experience with them. I like how I can read service behaviours in test logs clearly and identify failure causes within a few lines of code. However, my current team have not been quite so enthused with the extra investment in complexity required to write them. I’ve started noticing developers ignoring my examples entirely and falling back into free-form tests, sometimes with framework abuse, other times dropping the framework entirely. I can’t really blame them as, I’ve come to realise that specification structured testing is over complicated.

It has taken me writing three testing themed posts to do so but I now believe that I write tests in more of an opinionated style than other developers. I think that this has been caused by some unforgettable past experiences of working with tens of thousands of lines of very badly written free-form tests that reliably violated fundamental programming concepts such as DRY. In my opinion, when fixing unit tests takes orders of magnitude longer than writing application code then your technical debt has reached a critical status. I remember getting away with deleting and disabling entire test classes in order to make my life easier. No-one reads the tests in pull requests right?

From this, I think I value test structure, readability and maintainability to a far higher degree than some of my peers, who seem to often consider testing as an afterthought. This has led me to believe that with specification structured testing, I may have engineered too much complexity for it to succeed as a framework. It needed more of a balance between capability, complexity and… as ridiculous as it sounds… beauty. Whilst specifications look great in logs, in code, specification inheritance can get so out of control that they become extremely difficult to read. I think developers prefer to write code that looks nice, feels cool or seems like a productive use of their time.

What’s Changed?

I recently introduced my team to a new fluent fixture based integration testing framework to very positive feedback. Admittingly, we were using Postman based tests before so anything would seem like a revolution, but in comparison to attempts coming from other teams, we were definitely on to a good thing. A fixture based approach was a perfect fit for integration testing as tests written for a particular API or entity type, will follow a very similar pattern.

Configure the test server

Configure the database, including adding any entities required for the test

Make an HTTP request

Try to deserialize the HTTP response as JSON

Assert against the response

Assert against changes in the database

The fixture reduces the boiler plate involved with bootstrapping and calling the test server, whilst the fluent extension based approach allows easy integration with libraries specific to each database technology used by each of our microservices. Most of all, it feels productive to use. With tiny but powerful extension methods like HavingDatabaseWithEntity, WhenMakingRestRequest and ShouldReturnJson, tests are readable and verbose without violating DRY.

Given Fixture

Fluent fixtures work great for integration tests but how about unit tests?

The structure of unit tests is not quite as standard as integration tests but there are still patterns that we can observe for writing a fluent process.

Generate test data

Configure mocks with the test data

Optionally construct a subject from the mocks

Call a method on the subject or a static method

Assert features of the result returned or exception thrown

Assert expected mock actions

Using this flow as a base, I have developed given-fixture, a really simple library that provides a fixture to configure and assumes a fluent style of writing tests.

Fluent convention

To develop a successful fluent style, we need a convention for using fluent verbs. For given-fixture I went with:

Having for arranging e.g. Having some data, Having some configured mock.

When for acting e.g. When we use a subject to call a method, When we call a static method.

Should for asserting e.g. Should return a specific value, Should throw a specific exception.

The name of each fluent extension should begin with one of these verbs. Since given-fixture is configured with a fixture, there is no strict ordering of fluent extension methods, but to keep a clean conversation flow we should always conform to the order “Having”, “When”, “Should” i.e. arrange, act, assert.

Generating test data

Some test data must conform to a particular structure in order for our tests to be valid e.g. the test subject might expect URL’s or email addresses to be parsed or validated. However, the vast majority of all test data is characterized by us not really caring about the value itself, just that the test subject used it in the expected way to produce the expected result.

For structured data, the excellent Bogus library is perfect. The test fixture in given-fixture provides a Faker instance so that we can wrap it fluently like so.

I noticed whilst writing tests using these methods that most of the time I was mocking methods to return models or throw exceptions that I had just generated with AutoFixture. So I have included a set of extension methods that cover most of these common cases. Using these extension methods, the above example becomes more concise.

Each of these methods calls the Verifiable(string because) method on the mock object. This is a good practice as it asserts all mock actions were actually completed i.e. the test is actually testing the subject behaviour that we expect. After result and exception assertions have completed, the test fixture automatically calls VerifyAll() on each configured mock.

The subject

Once we have all test data ready and all dependent calls mocked out, we need a subject to test. With given-fixture, you can use Autofac.Extras.Moq to automatically construct a subject using it’s mock repository and the Autofac IoC container. If you are testing instance methods, this is the preferred approach as changes to the constructor signature will not automatically break all tests. The fixture also allows testing without a subject in order to test static methods.

Assertions

For asserting features of the result, the fixture provides ShouldReturn and for the thrown exception, ShouldThrow. The library also provides a selection of common assertion extension methods using the excellent FluentAssertions library.

The fixture will throw if you attempt to use the incorrect run method for your act step.

Full example

Imagine that we have BreakfastService, a service for creating breakfasts. This is simply naming and pricing collections of breakfast items.

publicclassBreakfastService{privatereadonlyIBreakfastItemRepository_breakfastItemRepository;publicBreakfastService(IBreakfastItemRepositorybreakfastItemRepository){_breakfastItemRepository=breakfastItemRepository;}publicasyncTask<Breakfast>GetBreakfastAsync(GetBreakfastRequestrequest){if(request==null){thrownewArgumentNullException(nameof(request));}// Make sure we have some breakfast items.if(request.BreakfastItems==null||!request.BreakfastItems.Any()){thrownewArgumentException("All breakfasts must have breakfast items",nameof(request));}// Get the breakfast items.varitemTasks=request.BreakfastItems.Distinct().Select(_breakfastItemRepository.GetBreakfastItemAsync);varitems=awaitTask.WhenAll(itemTasks);// Ensure we have all items.if(items.Any(x=>x==null)){returnnull;}// Make the breakfast.returnnewBreakfast{Price=items.Sum(i=>i.Price),Name=GetBreakfastName(items)};}privatestaticstringGetBreakfastName(ICollection<BreakfastItem>items){varitemTypes=items.Select(x=>x.Type).ToList();// Check for full english breakfast.varisFullEnglish=Enum.GetValues(typeof(BreakfastItemType)).Cast<BreakfastItemType>().All(itemTypes.Contains);if(isFullEnglish){return"Full English Breakfast";}// Check for breakfast items on toast.if(itemTypes.Contains(BreakfastItemType.Toast)){varnotToast=items.Where(x=>x.Type!=BreakfastItemType.Toast).ToList();vartoast=items.Except(notToast).First();return$"{GetItemNames(notToast)} on {toast.Name}";}// Fall back to a list of all items.returnGetItemNames(items);}privatestaticstringGetItemNames(IEnumerable<BreakfastItem>items)=>Regex.Replace(string.Join(", ",items.Select(i=>i.Name)),",(?=[^,]*$)"," and");}

First of all, each test will share common test fixture configuration steps, such as configuring the subject and method to call. To avoid repeating ourselves in code we should wrap these in extension methods.

internalstaticclassBreakfastTestExtensions{/// <summary>/// Configures the mock breakfast item repository to return a relevant breakfast item/// when called with the specified breakfast item type./// </summary>publicstaticITestFixtureHavingBreakfastItem(thisITestFixturefixture,BreakfastItemTypetype,outBreakfastItemitem)=>fixture.HavingMocked<IBreakfastItemRepository,BreakfastItem>(x=>x.GetBreakfastItemAsync(type),outitem,c=>c.With(x=>x.Type,type).With(x=>x.Name,type.ToString()));/// <summary>/// Configures the fixture to construct a <see cref="BreakfastService"/> subject/// and call <see cref="BreakfastService.GetBreakfastAsync"/> with the specified breakfast item types./// </summary>publicstaticITestFixtureWhenGettingBreakfast(thisITestFixturefixture,paramsBreakfastItemType[]types)=>fixture.When<BreakfastService,Breakfast>(x=>x.GetBreakfastAsync(newGetBreakfastRequest{BreakfastItems=types}));/// <summary>/// Configures the fixture to assert that the subject returns a breakfast with the specified name/// and price as calculated from the specified breakfast items./// </summary>publicstaticITestFixtureShouldReturnBreakfastWithCorrectNameAndPrice(thisITestFixturefixture,stringexpectedName,paramsBreakfastItem[]expectedItems)=>fixture.ShouldReturnEquivalent(newBreakfast{Name=expectedName,Price=expectedItems.Sum(i=>i.Price)});}

Our first tests should assert that relevant argument exceptions are thrown when calling the service with bad arguments.

Finally we should write tests for behaviours that are not represented by the happy path i.e. code in the service that has not been called by any of the above tests. Firstly we need to test the failure when one of the breakfast items cannot be retrieved from the repository.

Next we have the case where all breakfast items are requested, then we should get a special case of a “Full English Breakfast”.

[Fact]publicTaskWhen_getting_full_english_breakfast()=>Given.Fixture.HavingBreakfastItem(BreakfastItemType.Bacon,outvarbacon).HavingBreakfastItem(BreakfastItemType.Egg,outvaregg).HavingBreakfastItem(BreakfastItemType.Sausage,outvarsausage).HavingBreakfastItem(BreakfastItemType.Toast,outvartoast).WhenGettingBreakfast(BreakfastItemType.Bacon,BreakfastItemType.Egg,BreakfastItemType.Sausage,BreakfastItemType.Toast).ShouldReturnBreakfastWithCorrectNameAndPrice("Full English Breakfast",bacon,egg,sausage,toast).RunAsync();

Next we have the case where not all Full English Breakfast items are requested but the selection includes toast, then we should get a special case of “{items} on Toast”.

[Fact]publicTaskWhen_getting_bacon_and_egg_on_toast()=>Given.Fixture.HavingBreakfastItem(BreakfastItemType.Bacon,outvarbacon).HavingBreakfastItem(BreakfastItemType.Egg,outvaregg).HavingBreakfastItem(BreakfastItemType.Toast,outvartoast).WhenGettingBreakfast(BreakfastItemType.Bacon,BreakfastItemType.Egg,BreakfastItemType.Toast).ShouldReturnBreakfastWithCorrectNameAndPrice("Bacon and Egg on Toast",bacon,egg,toast).RunAsync();

Finally, we need to make sure that the service is reliably deduplicating breakfast items. TODO: support multiple items e.g. 2x Bacon.

Final Thoughts

So what do you think? Your testing opinions may not align perfectly with mine but surely you consider the idea of using a fixture to fluently test behaviours like this to be damn cool. I think a lot of developers already think in this fluent style, I’ve even seen methods named with fluent verbs being used half baked in free-form unit tests, so maybe providing a common structure such as given-fixture will reliably help you and your team to write clean, reusable and highly maintainable tests.