Don't Test That: 3 Ideas to Avoid When Split Testing Your Pages

There are generally two reasons marketers conduct split tests. The first one is continual site improvements - getting more conversions, lowering the bounce rate, and so on. The second reason is to get data so that you can tell Randy from management to “shut up about your rotating carousels already.”

As a marketer, you generally want to focus your tests on continual site improvements. (sigh) Management rebuttal is required every now and again (and let’s face it - fun!) when you’re trying to avert disaster, but over several cycles, it’s less productive than testing for site gains.

So if you’re testing for the right (and less fun) reasons, there are generally things you can skip. After all, your conversion lifts are only going to be as good as your hypotheses for your site. You can test these when they are unavoidable, but you generally want to stay very far away from these ideas.

Rotating Banners

So … start with the obvious, right?

A lot of landing pages have rotating carousels, and there’s been significant testing devoted to this.

Pages with rotating banners suffer from banner blindness : users generally ignore things that look like ads, so they are more likely to skip rotating carousels altogether. Even for the small number who pay attention, though, there’s usually not enough time or control to get through the content before the banner auto-rotates.

Broad Intent, Deep Page

All right, this one is generally for SEOs who are trying to hit a large set of search terms using one page, or site creators or designers trying to make a deep page fulfill several tasks.

At the top layer of your site, it’s okay to have a page cater to a larger set of tasks. From the top layers of your site, it’s okay to have research, price-comparison, purchasing, and educational tasks all laid out as options. As a user gets deeper into the web site, the idea is that they will go from broad to specific - and the deeper you get into the site, the more specific the intent is that the page caters to.

Every now and again, you see product or service pages that violate this. Those pages try to do everything in one go. The result is that you have a ton of deep-layer content, a slew of calls-to-action, and not enough attention being drawn to the key task of the page.

If you’re testing pages like these, you’re starting with a handicap. You generally want to test pages higher up in the information architecture for broad tasks, or deeper into the information architecture for specific tasks - don’t mix and match.

Long Mobile Forms

While there’s something to be said for the greedy marketer who is pinpointing the exact amount of pain a user is willing to tolerate on a mobile device before he or she quits, you generally want to steer clear of this tactic.

If something should not require a form, give it away without the form. If something requires a form on a mobile device, make sure it’s pretty short. Even when it’s pretty short, you should help your users out and provide the option for auto-complete.

After you have all of that down, then you can split test for titles, the absolute required fields, and so on. But you have to get the basics right, first.

Split Testing For Wins

You should be testing a lot of ideas. Still, starting with things that have either repeatedly shown to lose in tests, like rotating carousels, or those that are just plain painful for users, like long forms, is bad for time - for the most part.

That having been said, when you absolutely need to, it’s a blast setting up a/b tests so you can say “Randy, enough with the form fields already.”

We at SiteTuners get it - it’s tough to build great mobile experiences on top of creating usable desktop designs. We respect many of the sites we will cite in this post greatly - we appreciate the sites that get even half of these things right.

We love quantitative data, but quantitative data can only take you so far. Google Analytics can’t tell you what the user’s primary goal is for visiting the web site. Site Catalyst can’t tell you what percentage of visitors can complete their tasks.

As much as we love analytics tools, we’d be the first to admit that you shouldn’t be using all your time on them. After all, time spent on Google Webmaster Tools, Site Catalyst, Google Analytics or OpinionLab can instead be used to solve issues found on Google Webmaster Tools, Site...

Online marketing tools today can either cost an arm and a leg, or not very much at all. The “not very much at all” doesn’t mean you’re settling for bad tools - there are great low-cost tools out there, just as there are bad expensive tools.