Your Simple Guide to A/B Split Testing

First of all, what is A/B split testing? In basic terms A/B split testing is a variety of marketing testing where customers randomly receive almost identical, yet slightly different, test samples.

This method can efficiently be used in various niches such as brochures, email campaigns and landing pages. A/B split testing can help you to improve conversion rates, better understand visitor behavior, test out new products and much more. Email newsletter services like Mailchimp have already implemented A/B split testing in their feature range.

There are plenty of useful tools already made like Google Website Optimizer which cost you nothing. Continue reading to find comprehensive guides on A/B split testing, exciting case studies, useful tools and resources.

Know What to Test

The goal of A/B split testing is to improve your website’s performance, not to build a whole website. Find out what exactly you want to test before testing it. Help out with case studies and have a closer look at your own website.

If you’re not sure start with a bit. Some of the most popular things to test are call to action buttons, headlines, forms, text length and layout.

Give it Time

Don’t jump to any conclusions after you’ve run the test for a couple of days. A solid A/B split test might even take a month. Giving up too early on a test will only cost you time and the results will be marginal.

Once you start seeing a solid path where the results are going you can close the test or start a new one. There are also calculators available to test results confidence if you’re unsure.

Analyze Results

You’ve run your first A/B test, great, now what? Don’t take it for granted that the test version is always a better solution. Examine your results carefully and see where the flaws were and what you could improve even more. One test often won’t be enough.

You’ll have to do several or even a multivariate one. Once you’ve done a decent amount of tests you’ll see where the actual results are going.

Mix it up

Like you just found out – one test won’t be enough. Be creative with your tests. Don’t stick to the ordinary A/B option, 50/50 audience. Create various templates and vary the proportions. Change something else in each test. At the end you’ll have rich and valuable material to analyze.

However don’t get too fancy. Don’t torture your regular visitors with too many testing. Address tests to your new visitors then see how they adapt. To sum up, follow the guidelines but don’t take them as a must. Be creative with some limitations though.

Articles & Tutorials

The Ultimate Guide To A/B Testing: This article by Paras Chopra, the founder of Visual Website Optimizer, is meant to be the best guide you will ever need for A/B testing.

Split Testing Landing Pages to Boost Conversions: Running split tests on campaign landing pages is an important way to test and optimise conversions. Learn how to use split tests and optimize landing pages.

Skype Boosts Homepage Conversion Rate with Radical Simplicity: Skype designed two new homepages to draw user focus to the call to action. See which was the most succesful option.

Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement: This post will show exactly how one start-up improved their homepage conversion rate (visitor to sign-up flow) more than 20%, then 16% again, with a few simple changes and Google Website Optimizer.

Tools & Resources

Visual Website Optimizer: Visual Website Optimizer is an easy to use A/B testing tool featuring point-and-click test designer and WYSIWYG editor for creating variations. If you aren’t sure if Google Website Optimizer will work for you this is definitely the second best choice.

Vanity: Vanity is an experiment driven development framework for Rails. It will help you to create and run A/B tests though it requires some heavy knowledge.

phpScenario: phpScenario is an object-oriented split testing library written in PHP. It’s designed to do simple A/B testing with as little code as possible, but should also be as powerful as you need it to be.

ShrimpTest: ShrimpTest is an A/B testing solution for WordPress under development at Automattic.