Enterprise Commerce Blog

Brought to you by

Webinar Recap: I Know I Should Be Testing, But…

Thanks to all who signed up and attended today’s webinar I Know I Should Be Testing, But… with Bryan Eisenberg. We did have some technical difficulties with capturing audio, but Bryan graciously agreed to re-record the webinar and you can find the replay here and even download the mp4 file to replay on your iPod or similar device. So big thanks and a hoorah for Bryan!

Without further ado, let’s jump into the summary of the original webinar, including the question and answer period…

No More Excuses!

Great experiences aren’t accidents: why we test

“…a computer can tell you down to the last dime what you’ve sold. But it can never tell you how much you could have sold.” – Sam Walton, founder of Wal-Mart

76.7% of retailers are not testing – how come?

We’re too busy dealing with urgent, we forget the important.

There’s a HiPPO in the room (will explain below)

IT/technology constraints, lack of human resources

Just plain don’t know where to start!

About the HiPPO: You may understand the value of testing, but are challenged in getting buy-in because there’s a HiPPO in the room – the “highest paid person in the organization.” Hippos are the biggest barrier to testing, according to Bryan, and sometimes you need to take initiative and show the results of testing in order to get more support for it.

All the barriers revolve around 3 key drivers:

TOOLS – Besides Google Website Optimizer (free) there are enterprise tools like Omniture Test and Target and Optimost. GWO may not be as robust as paid tools, but at least no one has the excuse they can’t afford a testing tool.PEOPLE – Like analytics, your tool is about 10% and the rest is people and process. You should have designated people working on testing so it doesn’t fall by the wayside when the “urgent” pops up.PROCESS – You can have the hands to do the work, but without a process for determining what to test, how and how to measure success and actually implement the improvements, you’ve essentially got monkeys with typewriters.

Challenges

According to a 2006 JupiterResearch/ERI Executive Survey of decision makers in companies currently using A/B or multivariate testing, the biggest testing challenges include (in order of importance):

Demonstrating ROI

Creating test elements

Modifying pages for testing

Establishing test scenarios

Using test interface

Prioritizing initiatives

Understanding results

Acting on test results

Segmenting/targeting test participants

Example of Tests

You can test any “section” on your website, here’s an example of 3 different headlines and 3 different images:

Amazon is famous for testing anything and everything, including cart buttons:

Bryan advises against any “slicing and dicing.” Don’t take a page and start testing every element on the page concurrently – it will drain your resources and the test duration will go on forever. Start with a headline, start with a cart button. Get some small wins and then test more.

Another mistake is to do testing, but not implement any changes. Commit to continuous improvement – plan, measure and improve. Take data, and make changes!

Process

Process for improvement:

1. Who are we trying to persuade?
2. What action do we want them to take?
3. What action to THEY want to take? (Not always identical to the action you want them to take!)

Step 1 – Focus on Your Customer’s Experience

Use these questions as your blueprint for testing:

How many different types of profiles will you have that would participate in this campaign?

Develop rich personas based on the 4 buying modalities – for more information on “persuasion architecture,” personas and buying modalities, check out Bryan’s book Waiting for Your Cat to Bark? and our webinar on the subject (video / text recap).

Do have the correct messages for each type of profile and their motivation? You will likely need more than one.

Do you have the correct message for each stage of their buying cycle? Are they early in the buying process, in the middle, or are they ready to buy?

Check out an example of how Competitve/Spontaneous buyers “see” a page (left) vs. Methodical/Humanistic buyers (right):

Typical behavior from a competitive/spontaneous user – only reads the top part of the communication and typically has shorter fixations. While methodical and humanistic users are more likley to read to the bottom of the communication and make longer fixations.

Opportunity Cost of NOT Testing

A great example is one of Bryan’s clients, Overstock (maybe you’ve heard of them?) who had a 92% bounce rate on this page:

The problem on this page was the “Kids Titles for Learning and Fun” banner. People thought they were searching only kids’ titles. Changing that banner led to a 33% reduction in abandonment rate, 5% revenue lift overall, not just for DVDs and $25 Million — that’s $68,000 per day. (You might want to show this to the HiPPO in your company!)

Start with function – make sure your site does what you need it to do. Make sure it’s accessible – the site’s not down, you don’t hit 404’s when you omit the www etc. This includes accessibility for the visually impaired. Then look at your usability (not to be confused with conversion) which may involve observing people physically navigating your site and performing various tasks. This is all pre-testing stuff.

Begin testing at the Intuitive stage – what can you do to reduce friction in the buying process? Then explore the Persuasive – do people understand what they are buying?

Setting Up Your Test

1. Create a Descriptive Name

Like what you're reading?Subscribe to our weekly newsletter.

Join over 20,000 ecommerce leaders who have subscribed and receive expert advice about the world of enterprise commerce.

Point of Action Assurance Test on Lead Form

Identify if this is site wide test or for a campaign

2. Define Your Goal

I want to increase conversion…

How will you measure this? What are the KPI?

3. How Will You Achieve your Goal?

What are you testing?

What are the variables?

What are the variations?

4. Define the Control

What is your prediction/hypothesis?

What are you basing that on?

5. Let the Test Rip

6. Measure & Analyze!

What did we improve or not?

What did we learn?

What do we do next?

What do we do next?

Sample Test Setup – Trust Element

Test Name: Confidence Building: Point of Action by Lead Form
Goal: I want to increase conversion by measuring total conversion rate of leads.
Does adding language about our privacy policy and response time improve lead form conversion?
Control: The control form is our current form (A).
Prediction: I predict the form with the detailed point of action assurance will convert best (B).

Simple Tests to Get Started

Headlines

Calls to Action

Re-order Content

Simple Graphics

Confidence Building Language

Improving Product Descriptions

Bryan and Future Now conducted a Retail Customer Experience Survey last year, and found 62% of top retail sites had only a brief blurb for a product description, and only 11% had exceptional product copy. Yet, a study by Allurent reports 67% of consumers who visited an online store intending to make a purchase left because the retailer did not provide enough product information.

The E-tailing Group found in it’s study:

77% of online shoppers are “very to somewhat” influenced by the quality of content (descriptions, copy, images and tools) when deciding to purchase from an online retailer

76% believe content is insufficient to complete research or purchase online “always,” “most often” or “some of the time”

When faced with incomplete information, 72% go to a competitor or research further

Got your attention?

Most people abandon because of lack of information to buy. But you can use customer reviews to dramatically improve product descriptions (GrokDotCom’s Holly Buchanan wrote about this, and so did we) and conversion. Amazon reviews and Buzzillions are great places to mine for this gold. Here are some tips:

Look for products with low look to book ratios & 3 -4.5 star reviews

Pull the “trigger words” from each review

Plot them as Logical or Emotional

Modify your product descriptions based on the results

And you can plot them on a chart like this if you like:

Here’s an example for a Kenneth Cole watch. Old product description:

Here’s revamped copy, taking in the sentiments of actual people from customer reviews:

Kenneth Cole Women’s Stainless Steel Watch

This unusual double chain bracelet band and watch is an instant attention getter

Kenneth Cole design features hypnotic mother-of-pearl face and is pleasing to look at

The high polish silver hands compliment the shimmery dial

No worries while washing hands, because this watch is water resistant to 30 meters

Influencing Your Organization

Step 1 – Get the Math Right

If 100,000 people visit your website, and 3% of people convert into a desired outcome – that’s 3,000 total conversions, right? If you say your test improved conversion by 1% – does that mean you know have 3,030 conversions or 4,000?

Make sure everyone understands what a lift means. Talk the language, you had a 25% increase in business if it was an increase in overall conversion rate from 3% to 4%, rather than 3% to 3.03%. That’s a BIG difference!

Step 2 – Take Control of the Visit

100,000 visits
x 3% CR
3000 orders

If you increase traffic (invest more in PPC, for example), without optimizing for better conversion, a 33% increase in visits only produces 990 more orders. If you increase conversion rate by 33%, with only 100,000 visitors, you make 1,000 more orders – and this benefit is permanent – where buying more traffic is a one time deal, and you have to continually buy more traffic.

Think about this:

1. You can’t always control the amount of visits, but you can control what you present to the visitors.
2. What would it cost you to double traffic (if possible) versus doubling conversion rate?

Step 3 – Demonstrate It!

Questions and Answers

Many ecommerce sites use dynamically generated pages? How do you test database driven sites?

Google Website Optimizer works as long as Javascript tags for the test are showing up in browser. It will produce the variables you input to the tool, not what is coming out of your database. More details in the book.

What is the minimum traffic you need to conduct an accurate test?

Keep variations small and look for something that has at least 100 actions per week – if not orders, make it add-to-carts. If you have no traffic it’s hard to do less than that.

If something is out of stock, should you remove the product or announce something is out of stock?

Bryan shares a story about a Sony camera that he loves. Recently at a conference he was sitting with a bunch of people discussing how great this (older) model of camera was. They did a product search on Sony’s website and 3-4 other websites, no one had the camera listed. All the rest were out of stock. Since no product pages exist, you can’t even write a review, and you certainly can’t buy what you can’t find! Bryan suggests rather a retailer should revise the page, explain that the model is out of stock and show options for a similar model. Even tell people that the new model retains the speed (or whatever the features that made the previous one a customer favorite) and is available and in stock now.

I wrote about this topic last month, how to use 301 redirects to preserve your link juice and SEO benefits of an existing product page. Jason brings up a good point that there’s a problem with redirecting visitors because you lose the “scent trail” – you don’t want to send people to a 190 model when they were expecting a 180. You never want to pull a bait and switch or cause user frustration. I’ll keep my eyes out for good examples of retailers who keep pages up for a future post.

Is it better to use a long landing page or multi-step process?

There’s no universal answer, it depends on who you are marketing to and what you do. Though Bryan is a fan of breaking up pages — it’s hard to find exactly what you’re looking for on a long page — you can also break it up badly too.

How do you build credibility for a new site or new brand? Without product reviews, what is the number one way of building credibility?

Have an About Us page to start, and “be real.” People know they are not on Walmart.com. Communicate where you excel in service, quality, unique products, clarity, great experience, better content than everyone else, et cetera. Keep testing and you’ll get better results. Don’t forget phone numbers and addresses, live chat, AIM — transparent / visible contact information.

What are ramifications of SEO and testing, is it a form of cloaking?

It’s not, really. Sure, you could use it for the wrong reasons — you must be careful of what you’re doing, you’re not trying to bait and switch. But from the SEO perspective, when a spider comes, it will spider your “control” page, it has no ramifications to SEO otherwise. If you have a radical improvement as a result of your testing and you implement it, you will have to modify your control which may affect your SEO.

Is there ever a reason you would have a successful test, then go back and retrace your steps and perform the identical test – perhaps at a later date, if customers change, economic conditions, seasonality, technology et cetera?

Yes, Amazon is constantly playing with order and formatting of text and product descriptions, for example. If something causes a significant lift in conversion, it makes sense you want to continue testing the same thing – perhaps in slightly different ways. Don’t retest for small improvements – 3% change, for example. But 15%, 20% or more, it’s worth further experimentation.