Over the last couple of years at Domain we have embraced Agile development and Continuous Delivery. The team is encouraged to merge development code into trunk and push small, incremental changes into production as early as possible. Code sitting in development branches is essentially stale and at risk of becoming redundant at any moment, so if branches are used it’s critical that merges happen as soon as possible.

Shipping to production regularly is great, but it’s also imperative that we provide a consistent and reliable experience to our loyal users & clients, else we risk losing them to a competitor. Additionally, we want to ensure that any changes we make to the product have a positive outcome.

With these two seemingly conflicting goals in mind, we started thinking about what methods we could use that would enable us to release all these code changes (shiny new features!) to users as often as possible, while minimising the risk in doing so. One of the methods we decided to invest in was feature flagging. There are a lot of great write-ups on how other tech teams achieved similar goals, such as Facebook’s Gatekeeper, Flickr’s Feature Flippers and Etsy’s config flags and many of these provided inspiration.

It became clear that we would need to separate the concept of deployments (pushing new code to production) and releases (enabling new features), and with that we set about building our own feature flagging framework.

Introducing Domain Switchboard

The Domain Switchboard is our solution to implementing feature flags. It’s the control center for all tests and new feature releases currently running, or rolling out on Domain. It looks a lot like this:

Adding a feature flag is as simple as specifying the flag name, and whether it is turned on or off. As of the time of writing we have 45 feature flags live in the production environment.

The flag name should be as descriptive as necessary, and is used within application code to check the status of a feature. This implementation is very straight forward. Here’s an example used to render two versions of our search results:

Feature Flagging Framework in use

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

@if(FeatureToggle.Enabled("MSiteNewSearchResults"))

{

<ul class="properties clearfix"id="ul-listings-results">

@foreach(varmodel inModel)

{

Html.RenderPartial("NewStuff");

}

</ul>

}

else

{

<ul class="properties clearfix fourteen"id="ul-listings-results">

@foreach(varmodel inModel)

{

Html.RenderPartial("OldStuff");

}

</ul>

}

Using this method we now have control at run time of which version of the search results is rendered to our users. One neat feature that was built into the framework is that if a check for a flag is made, and a flag with that name doesn’t exist in the database, it will automatically create the flag and disable it. This makes it safe to push code through development, UAT and production environments without having to configure flags in advance.

This approach works really well for turning features completely on and off for all users. A time it is specifically useful for Product Managers is when they are trying to co-ordinate releases across multiple platforms, or with another external event like a press release going out.

But .. what if we only want to turn on features for a subset of users?

Switchboard Rules

One of the core motto’s we live by at Domain is to Ship Early and Ship Often. We believe that the best way to get real feedback on new features is to put them in front of users and measure the result. As our feature flagging framework matured, we started to think of ways we could turn on features for just a segment of our audience in order to gather feedback and measure the impact of changes in a more controlled way. This lead us to develop the concept of switchboard rules.

There are a few rules that the team has found the most useful:

Percentage

This is fairly straightforward – the flag will turn on for a configured percentage of users. In order to give users a consistent experience a cookie is dropped to track weather they have the feature turned on or not. All going well, the percentage will slowly get ramped up to 100.

Logged In Status

Another way we can segment our users are based on whether they are logged into the site or not. Users that are logged in are typically more highly engaged and we may like to run tests that specifically target them.

Specific Users

For a really controlled rollout we can enable features only for specific users. This seems like it could be a little restrictive, however it is extremely useful for internal testing and to gain stakeholder feedback from interstate teams or customers.

However, there is another possible use for this setting. Strava have recently given the ability to users to opt-in to experiments of their own accord. The concept is not too different to “Labs” in Google products. Wouldn’t it be great if one day our experiments were mature enough to allow end users to opt-in to the latest enhancements on Domain? It would simply be a matter of programmatically adding their username to this list.

User Groups

In our membership system we have the ability to put users into groups – perhaps we could create a group of all the internal Domain staff, or a particular set of real estate agents, or users we have somehow defined as power users. By using the group name we can enable features for this predefined subset of users. This is an effective way of dogfooding our product changes.

Other Ideas

There are other segments we have discussed but never had the need to implement yet. For example it may be useful to segment users based on their browser, their physical location or whether they have performed a particular action on the site. There are many possibilities and the framework can be upgraded as they are needed.

Feature Flags Vs A/B Tests

At this point the line is getting blurred between what you may call a feature flag, and what you may call and A/B test. Feature flags are useful from a technical perspective to ensure that a features performance is adequate or that it isn’t riddled with bugs, but it is also useful from a product perspective to test that the feature is achieving the desired outcome. It is not uncommon that one of our features reaches production only to be turned off again for further development and tweaking.

One way to look at feature flags is that when they are turned off, it is basically an A/B test where B is weighted at 0. With that in mind, it actually makes sense to combine these two ideas into one framework.

Further Enhancement

API-First

This framework was built in the early days of our platform re-architecture and was unfortunately baked into the desktop website. This has meant we ended up with slightly different implementation on our mobile applications. These days we build everything API-first so we can target whatever platform is most suitable. Very soon we will need to wrap up this logic in a microservice and consolidate the mobile applications.

Date Since Last Change

The reality with feature flagging frameworks is that they add additional complexity to your source code in the form of switches, typically if statements. Without attention these can be left behind and really start to pollute your applications. One enhancement we have in mind is to simply display the length of time since the flag was last modified. This would give a good indication which features have matured and should be cleaned up.

The investment was not that high actually. I’d say we spent about a week or so implementing the framework, and just under 2 weeks building the dashboard.
We are looking at open-sourcing a few of our core components, and this is definitely one of them. We’ll post on the blog when we do, so stay tuned!