Using Optimizely for A/B Testing

What is Optimizely?

Optimizely bills itself as the world’s leading experience optimization platform. It allows you to directly edit different aspects of your website and test the variations against one another (exactly what you’d expect an A/B testing tool to do). This comprehensive tool allows you to broadly optimize the visitor’s experience with your site, ensuring that you wind up with the best possible content for converting visitors to paying customers.

When used properly, Optimizely squeezes more juice out of a site’s existing traffic, improves important success metrics, and ultimately drives more revenue. After a relatively simple registration and setup process, you can be on your way to testing different variations and measuring success using detailed metrics. It won’t come cheap if you want to get the most out of it, and it won’t catapult you over all of your A/B testing hurdles, but it’s definitely a tool worth considering. Let’s take a look at how it works, how to get your first experiment started, some key features and ultimately why Optimizely still leaves the would-be tester wanting more.

How does Optimizely work?

The wonder of the tool is that for all of its capabilities in facilitating experimentation, it is rather straightforward (at least in the beginning). There’s a slight learning curve since it has multiple capabilities, and it will take some time to get used to the functionality and terminology, but for a simple experiment, it’s quite easy. Once you’ve signed up, you enter the URL of the page that you’d like to optimize into the URL box. From there, you are now free to edit any aspect of the content that you’d like (from within the browser, not Umbraco). For instance:

Web content

Cosmetic styles (colours, for example)

Layouts

A non-technical explanation of the inner workings of the tool is that, after kicking in by using just one line of code on your site, it changes the experience for visitors depending on which variation they’ve been bucketed into (the % of traffic that you’ve selected to see each variation). It then collects data, such as:

Pageviews

Conversions

Engagement (the % of visitors who clicked on any part of the experiment page)

Or even custom goals like revenue (more on goals later)

Once you’ve selected the web page that you’d like to edit, it appears in the tool exactly the same way a visitor would see it on the web. No coding experience necessary. It makes for a simple and intuitive experience for the editor due to the pleasing interface, but the visitor might take issue with the fact that once they’ve typed in the URL of the page they’ll see the original page, only to have the variation load once Optimizely’s code kicks in. This hiccup might put some visitors off, and so might the prolonged loading times for slower browsers. Don’t worry, there’s a solution for that. We’ll see about that after we get our experiment off the ground.