Scattered postings about all things code related

Main menu

Post navigation

As mentioned in my previous post I am trying to experiment with more resilient ways to perform test driven development. In my ideal world, not changing functionality means tests do not need to be changed.

Recap On Unit Testing Shortcomings

Unit testing too often breaks with refactors unless the refactoring only takes place inside a single class. More often than not you start writing code to do something in one class and then later realize you need the same logic in another class. The proper way to handle this is to extract the logic into a common dependency (whether it is a new dependency or existing one), but if your tests are scoped to a single class (with all dependencies mocked) this becomes a breaking change requiring all of your mocks to be touched up (or new ones added) in order for your test to confirm that nothing has changed.

The problem with that is you are forcing the change in your tests to verify that functionality hasn’t changed, but if you are changing your test you can’t be 100% sure functionality hasn’t changed. The test is no longer exactly the same. All you can be 100% sure about is that the updated test works with the updated code, and then you pray that no regressions come out of it.

Expanding the scope of what you consider a unit for test purposes helps alleviate this, but that only works well if you have a complete understanding of both the code and business domain models, so that each unit becomes a domain. This is hard to do until a product is out and in active use because it is rare the business or engineering sides have a full grasp of the product before it is in customers hands. The product always changes after the MVP.

So What’s a Solution?

Scenario based testing is a test pattern I am playing with that tests the end user boundaries of a system. At the end of the day what I care about as a developer is that end users (whether they are people or applications) can perform actions against the system and the correct processes are run.

This means that if I am using a web based API and my system allows users to edit a blog post, my tests will verify that sending the correct data to my API and then retrieving that post back will return the post with the correct fields edited in the correct way. I don’t care about testing that it’s stored in the database in a specific way, or a specific class is used to process the logic on the post, I only care about the end result. If the results of an action never leave the system, does it really matter if it occurs?

So It’s An Integration Test?

Some of you are probably shrugging your shoulders thinking I’m talking about an integration tests. While these can somewhat be considered as integration tests, I try not to call them integration tests as all external components are mocked out.

Most integration tests involve writing to actual external data stores (like SQL) in order to verify that it is interacting with those systems properly. This is still necessary, even with scenario testing, because you can never be sure that your SQL queries are correct, that your redis locks are done properly, etc…

The purpose of this style of testing though is to codify the expected functionality of your system with as much area coverage as possible, lowering the needed scope of the required integration testing to just those external components.

Mocking all external systems also has the side benefit of being able to run tests extremely quickly and in parallel.

What Do These Tests Look Like?

Another goal for me in my testing is that it’s easy to read and understand what is going on, so if a developer breaks a test he has an understanding of the scope of functionality that is no longer working as expected.

If we take the previous example of editing a blog post, a test may look like this:

In this you can clearly see the functional intent of the test. As a developer you are able to completely change around the organization of the code without having any impact on the passing or failing of this test, as long as it is functionally the same as it was before your refactor.

If this looks like magic, that’s because it does require some up front development costs. The TestContext class mentioned above is a class that holds details about the current test, including IoC functionality and other information it needs to track to perform tests.

In my current code base utilizing this pattern, the TestContext class is initialized in the test class itself, and has all the logic for how to set up IoC, what it needs to mock, and what events in the system it might need to tie into in order to be able to perform verifications.

All the methods shown on the TestContext object are merely extension methods that utilize the test context to make it easy to perform common actions. So for example, SetDate(date) call would get the mocked date provider implementation from IoC and force it to a specific date (to allow freezing time).

`SubmitNewPostApiRequest()` could be something that either calls on your API using a fake browser or calls your API controller, depending on what framework you are using. The current code base I use this for uses Nancy for it’s API layer, which provides tools that allow sending actual POST or GET requests to the API for testing purposes.

What Are The Cons?

This approach isn’t perfect however. Since we are only testing the system boundaries it requires the developer to have a decent grasp of the underlying architecture to diagnose test failures. If a test starts failing it may not be immediately clear why a test is failing, and has the possibility to waste time while trying to figure that out.

This does force the developer to write smaller pieces of code in between running tests, so they can keep the logical scope of the changes down in case of unexpected test failures. This also requires tests to be fast so developers are not discouraged from running tests often.

All in all, I am enjoying writing tests in this style (so far at least). It has let me do some pretty complicated refactors as the application’s scope broadens without me worrying about breaking existing functionality.

Test driven development has proven itself to be a good thing for me over the years. It has helped me to not only codify my design decisions, but it has also helped keep me focused on the next iterative steps I need to create the larger change in the system that I originally set out to do. It has also helped keep regressions from occurring.

Unit testing is one integral part of test driven development, as it allows you to take a unit of code and make sure it does what it is supposed to.

What Is a Unit?

What qualifies as a unit of code differs from person to person. Most “conventional wisdom” of the internet will lead you to believe that in object oriented programming languages, a unit is a single class. However there is no reason that this should be the case, as you can have functional units in your code base that involve multiple classes or even a single function, depending on the full scope of what is being performed (and how logic is related to each other). The reality of the situation though is that as you start involving more classes in your functional unit, your tests start taking on a profile that looks more like integration tests rather than unit tests. This complicates the effort required to setup a test and seems to make me gravitate back to smaller units, which usually ends up back as a class as a single testable unit.

Unit testing at the class level is also considered good as it makes it much quicker to figure out where the code is that is causing a test to break, as the area of code that can be causing a bug is isolated to a single class (which should be relatively small anyway).

However, as time goes on I have started disliking testing with classes as the boundary of the test.

An Example

Let’s say we are writing code that contains the business logic for updating a post on a blog. A minimal version would contain several business processes, such as:

Verify the logged in user has access to the post

Get the current version of the post (with categories that have already been added to the post)

Get the list of categories known by the system

Modify the post with the new details

Save the post to the data store

Assume each bullet point is it’s own class or function that already has tests for them. I now need to test the class that is combining each of these components into a cohesive pipeline that performs the correct operations under the correct conditions in the correct order.

So let’s say I want to create a test that a user with access to a post can modify the title of a post. At the outset it sounds like a simple case of inputs and outputs.

Unfortunately, it gets a lot more complicated than that. Since i just want to focus on the functionality of my EditPost() function (since all the other functionality should already be tested and I just need to compose them together) I need to create mocks so that I don’t have to test the implementation details of checking if a user has access to a post, can retrieve a post object successfully from the data store, etc…

So for this test I create my first mock that says that when I pass my user id and post id to the authorization function, it should always return true.

The next mock I need to create says that when I pass a post id to the post retrieval function, it returns to me a post object with some pre-defined information.

Another mock I need to create says that when I call on the function to retrieve all blogging categories, it returns to me a static list of categories.

Yet another mock I need to create says that when I call the function to save the post to the data store, it siphons off the parameters I passed into it for verification.

Now that I have those mocks setup, I can now pass them into my edit post function call (via dependency injection), call my edit post function with the new title, and verify the parameters I siphoned off are what I expect them to be. If I am following the TDD methodology then I now have failing tests, so that I can fill in the function’s implementation and get a nice passing test.

You continue on creating more tests around this function (and all the mocks that come with it) to test authorization failures, post doesn’t exist conditions, other specific editing conditions (e.g. the creation date must always remain the old value on published posts, edit date is automatically set to the current date/time), etc….

So What’s the Problem?

So a few days later you realize that always pulling down a post with it’s full content and categories is too much. Sometimes you only need titles, sometimes titles and categories, sometimes titles and content without categories. So you split them out into different functions that return different pieces of the blog post (or maybe you have different functions for blogPostWithContent, blogPostWithCategories, etc…).

Or maybe you realize you need to add some more parameters to the authorization method due to changing requirements of what information is needed to know if someone is authorized to edit a post in some conditions that may or may not be related to the tests at hand.

Now you have to change every test of your edit post function to use new mock(s) instead of the previous mocks. However, when I wrote the original test all I really wanted to test was that an authorized user could change the title of a post but all my changes are all related to the implementation details of how that is accomplished.

The tests are now telling me that I have a code smell, and that my tests on the logical flow of the process are too intertwined with the tests surrounding how I’m composing all my functional units. The correct way to fix this problem is to move all the the business logic of how to modify a post object into it’s own pure function. This function would take in a data store post object, the change operations to perform on it, and the categories listing. It would then output the resulting post that needs to be saved to the data store.

We can then have tests on the logic of editing a post on the new function and keep the tests on the original function focused solely on how all the functions are composed together. Of course we have to completely modify the previous tests now to reflect this refactoring effort, and since the old tests are no longer valid we have to be careful to create regressions since we are completely changing old tests and adding completely new tests.

The question is, was this refactor worth it? At an academic level it was because it gave us more pure functions that can be tested independently of each other. Each functional unit is very small and does only one thing in a well tested way.

However, at a practical level it thoroughly expands the code base by adding new data structures, classes, and layers in the application. These additional pieces of code need to be maintained and navigated through when bugs or changing requirements come down the pipeline. Additional unit tests have to then be created to verify the composition of these functions are done in the correct and expected manner.

Another practical issue that brings this to a wall is that you are usually working with a team that has various skill and code quality concern levels, combined with the need to get a product and/or feature out to customers so it can generate revenue and be iterated on further.

It also increases the need for more thorough integration tests. An example of this is if the function that actually performs the edit requires the post to have a non-null creation date, or only is allowed to edit non-published posts. This function will have unit tests surrounding it to make sure it surfaces the correct error when it is passed data that it considers bad, but there is no way to automatically know that this new requirement is not satisfied by callers of the function with unit tests alone, and this error will only be caught when the system is running as a holistic unit.

So what’s the solution? There really is no good solution. Integration tests are just as important as unit tests in my opinion, but having a comprehensive set of unit and integration tests that easily survive refactoring is extremely time consuming and may not be practical to the business objectives at hand.

Integration tests by themselves can usually withstand implementation level refactorings, but they are usually much slower to perform, in some cases are completely unable to be run in parallel, require more infrastructure and planning to setup properly (both on a code and IT level), and they make it harder to pinpoint exactly where a bug is occurring when a test suddenly fails.

Every time I displayed this view the drop down was always on the first item and it was driving me nuts. After playing around in Linqpad I came across a clue:

0.250m.ToString(); // Outputs "0.250"

I stared for a while and finally noticed the trailing zero in the displayed output. I then made sure I wasn’t totally crazy so I tried:

0.250m.ToString() == 0.25m.ToString() // outputs False

I looked in my real database where the value was coming from, and since it is a decimal(18,4) entity framework is bringing it back with 4 decimal places, which means it’s including the trailing zero. Now it makes sense why Asp.net MVC’s helpers can’t figure out which item is selected, as it seems like a fair assumption that it is calling ToString() and doing comparisons based on that.

While trying to figure out a good solution I came across this StackOverflow answer which had the following extension method to normalize a decimal (which removes any trailing zeros):

After normalizing my model’s value, the select list worked perfectly as I expected and the correct values were picked in the drop down list.

Of course this is a clear hack based on implementation details and I reverted this fix. There’s no telling if or when a new version of the .Net framework may change this, and from what I’ve read a lot of the internal details of how decimals work are different in Mono, and this hack does not play nicely in Mono (supposedly in mono, 0.1250m.ToString() does not display trailing zeros).

The proper way to resolve this situation is to force the drop down list to do a numerical equality to determine which item should be selected, by manually creating a List instead of using the SelectList() constructor.

Why?

So even though I knew what was going wrong, I was interested in why. So that stack overflow answer pointed me to the MSDN documentation for the Decimal.GetBits() method. That link contained specifications for how a decimal is actually constructed. Specifically that internally a decimal is made up of 4 integers, 3 representing the low, medium, and high bits in the value and the last integer containing information on the power of 10 exponent for the value. Those combined can give exact decimal values (within the specified range and decimal places allowed).

So to start with I tried the decimal of 1m and printed out the bit representation:

So how does it convert a 10 into a 1? Looking back at the MSDN documentation it says that the first 15 bits are always zero (as they are in this case) with the next seven bits being the exponent of the decimal. Bits 16-23 are 0000001 and the last bit is zero, giving us an exponential value of positive one. These mean that to get the final value we take the value of the low + mid + high bits combined (10) and divide them by 10 to the power of positive 1. This gives us a value of just 1,

So now it’s clear that when you create a new decimal it essentially keeps track of the exact representation it was originally invoked with, including zeros and is pieced together to it’s fully realized value on a ToString() call.

Recently I had the idea of a log parsing and analysis system, that would allow users (via a web interface) to define their own regular expressions to parse event logs in different ways. The main purpose of this is for trend analysis. Logstash is an open source project that does something similar, but it relies on configuration files for specifying regular expressions (and doesn’t have other unrelated features I am envisioning).

One issue I have with regular expressions is they can be very hard to create and maintain as they increase in complexity. This difficulty works against you if you are trying to create a system where different people with different qualifications should be able to define how certain logs are parsed.

While looking at Logstash, I was led to a project that it uses called Grok, which essentially allows you to define aliases for regular expressions and even chain aliases together for more complicated regular expressions. For example, if you needed a regular expression that included checking for an IP address, you can write %{IP} instead of (?<![0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9]). This makes regular expressions much easier to create and read at later on.

The problem with Grok is that it is written in C as a stand alone application. This makes it hard to use in .Net applications without calling out to the shell, and even then it is cumbersome to use with dynamic aliases due to having to reconfigure config files on the fly.

For those reasons, I created an open source library I called RapidRegex. The first part of this library is the freedom to generate regular expression aliases by creating an instance of the RegexAlias class. This class is used to give you full flexibility on how you store and edit regular expression aliases in any way your application sees fit, whether it’s in a web form or in a flat file. It does however come with a class that helps form RegexAlias structures from basic configuration files.

As a simple example, let’s look at the regular expression outlined earlier for IP addresses. With RapidRegex, you can create an alias for it and later convert aliased regular expressions into .net regular expressions. An example of this can be seen with the following code:

What makes this even more powerful is the fact that you can chain aliases together. For instance, an IPv4 address can be defined as 4 sets of valid IP address bytes. So we can accomplish the same above with:

In my random desire to build a simple IRC server I decided the first step was to build a TCP server that can read data and send data simultaneously to multiple users. When someone first starts to write a networked server they quickly realize it’s not as easy as they had hoped due to all the network operations being done via blocking method calls. Before .Net 4.5, handling multiple socket connections meant having to put the socket handling calls into alternate threads, either by AsyncCallback calls or manually loading the method calls into the thread pool. These then require thread syncing when data needs to get back to the main thread so that the server’s business logic can run on the socket events.

With the introduction of the Async/Await keywords in .Net 4.5, it has become much easier to run your socket operations asynchronously while never leaving the main application thread. If you are not familiar with the Async/Await keywords it would probably be a good thing to read Stephen Cleary’s tutorial.

So here is how I went about making a single threaded TCP server that can handle multiple clients. This may not be the best way, but it did seem to be the most successful attempt I had for this.

Network Clients

The first thing I needed was a class to handle already connected clients. I needed the class to be able to take an already established connection and have it be able to receive network messages (and pass them to the business logic for processing) as well as sending messages back to the client. It also needs to determine when a connection has been dropped (either gracefully or not). Most importantly, these actions should not block the general server flow. As long as the server’s business logic processor is not actually processing a network message, it should not be waiting on a networked client.

Since both receiving messages from the client and a client being disconnected are important aspects that the underlying server should be aware of, I defined the following delegates:

This handles creating the client (not the connection, but the class itself) and a helper method for handling when a client has been disconnected. Now we need to listen for incoming data on for this TCP client. Since reading TCP input is blocking, we need to perform this asynchronously with the following code:

This method essentially loops until the network client is not activate anymore and awaits for a full line of data to be returned by the client. If a null response is returned or an IOException occurs, then that means the connection has been disconnected and I need to mark the client as such. By making sure we are awaiting for incoming data, it assures we do not block the application flow while waiting for data to come down the pipe. This method returns a Task, so that we can check for unhandled exceptions in the main server.

Next we need to be able to asynchronously send data to the user. I do this with the following method:

The _networClientRecieveInputTasks would be used to check for exceptions while listening for input from a client. The client listen task will be used to reference the asynchronous task that listens for new client connections, and this would be use this to check for unhandled exceptions being thrown. Everything else is to get data ready for actually running the server.

We need to consider what to do when a command is received by the client. To get up and running quickly we are just going to relay the incoming data out to all other clients, via the following method:

The next thing we need is to figure out what we want to do when a client is connected. When a client connects we need to create a NetworkClient instance for them, assign them an identification number (for internal use only), hook into the NetworkClient’s events, and start listening for input from that client. This can be accomplished with the following method:

This will take a TcpClient, create a new NetworkClient for it, tie up the events and start receiving input.

We now have everything e need to execute information from a client, we just need to actually accept incoming client connections. This of course needs to be done asynchronously so we do not block the server flow while waiting for a new connection.

We need the while() loop due to the main functionality of the server running asynchronously. Otherwise the program would immediately exit. Now if you run the server you will be able to connect multiple telnet sessions to each other and pass messages back and forth.

Threading

The problem with our server so far is that it is not operating in a single thread. This means that once you have a lot of clients connecting, sending messages, etc.. you will come up with syncing issues (especially once you start adding real business logic to the mix).

Outside of WPF and Winforms applications async/await operate in the threadpool not in the main thread. This means you cannot 100% predict which await operations will work on which threads (you can read more about this at this MSDN blog).

If you want proof of this on your sample server, you can add the following code everywhere you have any other Console.WriteLine() call:

If you also add this to your Main() method and run the application you will notice multiple thread ids being displayed in the console.

Async and await commands utilize the SynchronizationContext which controls how (and on what threads) the different actions are run on. Based on that reference, I created the following SynchronizationContext implementation.

Now if you run the application and send some commands to the server you will see everything running asynchronously on a single thread.

Conclusion

This may not be the best approach, and a single-threaded TCP server is probably not the most efficient in a production environment, but it does give me a good baseline to work with to expand out its capabilities.

Recently I have been trying to find ways to reduce how much HTML I have to duplicate in my views and how I have to remember what css classes to give each set of divs. The problem comes with that the HTML that my views require don’t fit in the main layout, due to the fact that content still comes after it and some elements are optional, and they don’t fit well in partial views due to how customized the HTML inside the area is.

An example of the HTML I was dealing with, here’s an example of one of my views

I started thinking of how I can increase my code re-use to make this easier to develop and maintain. While looking over the view my eyes gravitated towards the Html.BeginForm() and I realized the most logical idea was to utilize using statements. So after looking at the implementation of Html.BeginForm() for guidance (thanks to dotPeek) I came up with the following class to implement the first few divs automatically.

This will write the beginning divs upon creation and the div end tags upon closing. After following suit with the other elements of my page and creating more disposable layout classes and Html helpers, I now have the following view:

Now I have a MUCH more maintainable view that even gives me intellisense support, so I don’t have to worry about remembering how css classes are capitalized, how they are named, what order the need to be in, etc…

While working with the LinkedIn API, I started becoming frustrated in testing my API calls. The core reason was that I couldn’t just form my URL in the web browser due to OAuth. In order to test my API calls I would have to write code to perform the call, test it out, use the VS debugger to retrieve the result to make sure the XML it’s returning is what I expect, etc.. It results in a lot of wasted time and I finally got fed up.

Introducing the ExtApi Tester

I developed a windows application to make testing API calls, especially API calls that require OAuth, to be done much simpler. I call it the ExtApi Tester.

The code can be found on GitHub. I’ve already gotten some good use out of it, and hopefully it helps someone else with their dev experience.

The code also includes an API for making it easier to call web API’s from code, but it requires some further refinement to handle the DotNetOpenAuth authorization process.