7 Reasons Your Site Isn’t Ready for A/B Testing

You’ve invested a lot of time and effort into perfecting your website and you want to get the maximum return from that investment. To achieve that goal, you’ve studied dozens of blogs on conversion optimization techniques. You’ve poured over countless CRO case studies, and you have a few tools to help you run A/B tests.

Before you start split testing to get those conversion gains, pause for a second. I don’t think you’re quite ready yet.

There are plenty of free tools to help you test your optimization – not to mention paid options from Optimizely to OptinMonster that’ll help you explore different facets of your site’s performance – so just about anyone can run A/B tests. But it’s not a matter of simply understanding how to do it.

The problem is that your site just isn’t there yet. A/B testing isn’t for everyone, and if it’s not done at the right time with the right conditions, you might end up accumulating a lot of false data that does more harm than good. Before you invest anything in testing and extensive optimization, consider these seven points:

1. The Traffic Volume Isn’t There

If this is what your traffic numbers look like, don’t bother A/B testing

There’s no doubt that A/B testing can be highly useful for businesses that want to improve their conversion rates. Having said that however, a lot of businesses shouldn’t bother with A/B testing.

Small businesses that are trying to grow, startups, e-commerce businesses in their early years and other micro businesses simply don’t have the traffic and transactions to accurately perform A/B tests. It takes a significant amount of traffic to provide accurate, measurable results.

You can see from this image that in order to detect a 10% lift, the tool recommends at least 51,486 visitors per variation.

If the traffic isn’t there yet, you can still optimize your site based on audience data you’ve gathered, but A/B tests won’t be helpful for a while and they might produce false information.

2. You Don’t Have Anything to Test

A lot of websites function as a general brochure for a company with minimal conversion points. If you run a B2B site or you have a freshly-created site with little more than a contact form and an opt-in, then it’s too early in the game to start running concurrent A/B tests.

If your site is content-lite then it’s probably too soon to start running tests.

Even if the volume of traffic is adequate to run accurate tests, you may not see a significant lift from a general opt-in or estimate request form. For most businesses, the amount of effort and cost that would go into designing variations for the tests just to get a small lift around micro conversions isn’t worth it.

The same applies to newer e-commerce stores.

Your time would be better spent with your analytics, where you can set up goal tracking, creating marketing campaigns, and developing your content offers and resources. The A/B testing can come later once you have more to offer and traffic has grown substantially.

3. You’re Not Sure What Matters

Before you can run any kind of tests, you have to understand what matters, because some elements are more important than others.

For example: a marketing agency is driving visitors to their estimate request page. They spend a significant amount of time optimizing that page with A/B testing variations and micro changes. After extensive testing, they find that their efforts made very little difference with virtually no impact on their conversions.

Another example: a brand selling shoes online puts a great deal of effort into optimizing and testing product pages, only to realize that the lift in conversion was insignificant. Instead, they could find ways to improve the average order value or review their funnel in Kissmetrics to find the biggest leaks where customers are dropping off and fix those problems instead.

If you randomly try to test what you think matters, then you’ll only be wasting time.

One study from Forrester showed that 60% of firms surveyed saw improvements in their website when they used a data-driven approach to design. It’s important to take the time to research what really matters to your business so you know what to optimize and where to make changes.

4. You’re Copying Content

Any tests you run after replicating their design and content will only be wasted. If the solution was as simple as copying what we thought worked well for our competitors (or even conversion case studies) then every e-commerce website would function exactly like Amazon.

The fact is, websites are highly contextual and they should relate to both the audience and what you’re promoting. Wal-Mart and Whole Foods are in the same business of selling food products, but they cater to completely different audiences and sell vastly different products.

If I stacked up my own services against another marketing agency offering identical services, there would still be contextual differences in how we market, how we service customers, the channels we use to engage them, and how we direct traffic to our sites.

You need to make sure your website is designed specifically for you, your channels, your audience, etc. before investing in testing.

5. The Data Isn’t There

The more capable you are with analytics tools like Kissmetrics or Google Analytics, the better off you’ll be. But, if the extent of your knowledge consists of checking traffic quantities, referral sources, time on page and bounce rates, then you’re only scraping the surface.

If you don’t know what data you need to monitor while A/B testing, then testing is a waste of time.

You have to approach your testing and analytics with a problem so you can find an answer in the data. That way, you can identify issues and confirm what aspects you need to change.

Learning a bit more about your analytics can tune you into:

How site elements or offers are performing

How your content is performing and whether it is keeping people engaged

What people are doing on your site and the routes they typically take

Where people are landing, as well as where they’re leaving

Where your funnel is losing money

The data won’t specifically tell you how to fix problems; it’s just a starting point where you can discover actionable insights. Without that data, and without the ability to interpret it, A/B testing is pointless.

6. Your Site Has Usability Issues

When was the last time you tested your website in a browser other than the one you typically use? Have you tried going through your entire site on a mobile device?

This is something a lot of marketers don’t consider when they start A/B tests. Ignoring usability issues, tech problems, and bugs is a huge mistake, though. Even minor bugs and slow load times can dramatically impact your conversion rates.

Just a one second delay in load time can drop conversion rates by as much as 7%.

You won’t get accurate results from A/B testing if segments of your audience are bailing due to usability issues. Some of your audience may never make it to your conversion point, and even if they do, their progress could be hindered by bugs or load times that will ultimately skew your results.

This misinterpretation could lead to changes and further variations of elements that are actually part of your winning, optimized design.

7. You Don’t Know Your Audience

Audience research should be one of the first steps of any marketing strategy. If your goal is to drive lots of traffic to your site with content marketing and paid advertising, I would hope you’ve done some measure of audience research.

Without it, you’re shooting blindly into the darkness and hoping to score a bullseye.

Without it, you’ll resort to guessing what to change about your copy, headlines, offers, and calls-to-action. Every variation you test will be just as random as the one before it, and you likely won’t see any significant change in performance.

Know who you’re marketing to before you make a large investment in A/B testing.

Testing isn’t for Everyone

While there’s a wealth of articles and advice online telling you test everything you do and to A/B test every variation, you don’t have to. For many statups and growing online businesses there just isn’t enough traffic early on to create an accurate sampling with measurable results.

Focus on growing your business for now. As you grow traffic levels, learn more about your customers, and targeted traffic increases you can start testing variations to go after those micro wins.

Do you use A/B testing on your site or landing pages right now? Have you found issues with the quality of your results? Share your thoughts with me in the comments below.