Opportunity-Out starts by finding the best areas of opportunity in your website, and working your way into a solid A/B or multivariate test from there. This is inverse of my preferred use of the Scientific Method for Conversion Rate Optimization Testing, but is excellent in helping crack a serious case of ‘strategists block.’

We’ll cover:

Knowing Where to run an A/B or Multivariate Test

Knowing What to Test in an A/B or Multivariate Test

Creating an A/B or Multivariate Test

Where to Run A/B or Multivariate Testing

A/B testing requires a large volume of traffic (or recipients) to yield statistically relevant results. With its additional variations, multivariate requires exponentially more data before it begins to yield insights.

Optimizely has a very handy A/B Test Sample Size Calculator which can help determine the number of visitors or subscribers needed for accurate test results. This calculator helps determine the sample size (visitors, email recipients) required per variation.

An A/B test is two variations.

A multivariate test is one variation per possible combination for each element. Testing a headline and call-to-action, two elements, at four versions each would be 4*4=16 variations.

Expecting a 15% lift on your 5% baseline conversion rate in a headline A/B test isn’t unrealistic, but telling your boss that 26,000 visitors will need to pass through the landing page before results are trustworthy might have you looking for a bigger impact test, or a higher trafficked opportunity for early tests.

What to Test in A/B or Multivariate Testing

Headline tests, subject lines, the addition of explanatory graphics or call-to-action button copy are popular starting points, as these allows highly visual test platforms for the best A/B test hypotheses.

Avoid the temptation to start in with button color, minor design modifications, and even in-copy tweaks. While these are great target areas for late-stage testing, the impact here is lower, requiring a larger investment in time for a smaller (if any) increase in conversions.

What is the most visual elements on your highly-trafficked opportunity from the step above?

In email, this is typically the subject line (for open rates), followed by the opening header and call-to-action for click-rate.

In a homepage, this is often a primary call to action in the navigation, ‘hero’ or banner area, followed by specific offers.

In a landing page, this is typically opening header, form and supporting imagery.

Offering a secondary offer can help engage those not yet ready to purchase.

Many in our email list would be interested receiving, and redeeming, an occasional offer.

It’s in the hypothesis that you’ll find your marching orders for your A/B or multivariate testing. Remember, your ‘A’ is your control, your current state. Your ‘B’ should include a test for your hypothesis.

Using this method opens the door to follow-up conversion-rate-optimization studies, such as ‘which kind of offer earns more engagement?’

Share and Swipe

I truly hope this helps you conquer your testing challenges! Pass this along to help make the internet a better converting place for all.