Elegant A/B Tests

Oct 15, 2018
-
4
minute read

Flexible software systems are a pleasure to modify. You know that you have a flexible system when you can change the behavior of the system without changing any existing code. Adding a new behavior should simply require adding new code. I will show you how to easily introduce A/B tests along with a new feature, without changing the existing code.

A/B Tests and Feature Toggles are common fixtures in a post-continuous-deployment software world. This makes it very important to know how to integrate them into your application in a clean way that doesn’t cause archtitectural instability. However, often they are implemented in a very clumsy and haphazard way, which then ensures that cleaning them up after resolution will also be rather messy and perhaps error prone.

The Conventional (Messy) Way

The conventional approach is to wedge them into the usage site procedure with some if/else logic, and a flag state provider, which might be nicely injected, or might be retrieved from a god-forsaken stateful static class somewhere.

This sort of design is horrendous! The code is messy and complex enough, and then – buried in the middle of the method – there is a surprise structural fork with an early return, and sometimes even an early exception thrown! Furthermore, even after the A/B Test is finished, cleaning up this code is going to require going through this class with a surgical knife and cutting out the correct bits of the code. It wouldn’t be hard to remove the wrong part of the feature, or leave in a residual dependency. Which parts of the setup are needed for the new feature? Which are for the old feature? What a mess!

Also, if the business wants to introduce another A/B test in a similar area of the application, this code site is about to become a Rube Goldberg contraption of conditional logic. At this point, you are thinking, there must be a better way!

Elegant A/B Tests

How can we have an elegant design that doesn’t require wedging in new conditionals into existing code? Build two new objects, and leave the original intact. One new object will be the new way to perform the business operation. The other new object will encapsulate the A/B Selection for a given customer.