Hello and welcome to The Marketing Blog. My name is Michael Fleischner and this blog has been developed to share my more than 14 years of Marketing experience with anyone interested in learning more about marketing or Internet marketing. The Marketing Blog markets marketing simple and covers all aspects of marketing with a particular focus on internet marketing.

Saturday, December 03, 2011

As someone who always dreaded "market research" and data, I have to say that over the years I've become much more appreciative of marketing metrics. Most importantly, measuring consumer behavior, online and offline, has radically improved my marketing results. Enter the concept of user testing. In today's guest post Jason Thai explains user testing and the more common pitfalls associated with it.

User-Testing Overview

Anytime a web-based business
releases a new feature, several issues pop up, keeping product managers tossing
and turning at night:

·1. Are
there any glaring bugs with the new feature?

·2. How will
customers view the new feature?

·3. Did we
design the feature in the most customer-friendly way?

·4. Does the
new feature impact other unidentified areas in the site?

·5. And most
importantly, how will the feature impact overall conversion?

If only there was a way to
test your products before launch… Luckily,
several companies have built fantastic user-testing platforms, providing companies with a pool of unbiased
respondents to provide feedback as if they were real customers. A lot of the sites offer great features such
as video walk-throughs with step-by-step user interaction. Another great aspect of user testing is that there
are few constraints on what you can test.
A few of the tests we’ve run include:

·Testing
preferences for before and after web
page designs

·Testing
the ease of our order flow process

·Testing
the usability of new customer features

Also, user-testing is
relatively inexpensive, making it a no-brainer for identifying pitfalls prior
to product launch.

What Happens When User-Testing Results Go
Against the Grain?

However, are the results of
user-testing always gospel? We, at
FoxTranslate, a certified translation service ran into that exact issue. A few months ago, we redesigned our homepage. We felt that the redesign more effectively
conveyed our key service features and more professionally segmented the content.

Current
Homepage

Redesigned
Homepage

To validate our intuition, we
ran user-testing to compare the two sites, expecting our redesigned homepage to
be the clear champion. Unfortunately,
the results of user-testing suggested otherwise.

User-Testing Results

If you needed a document translated,

which page would you more likely use?

Common Themes for Picking the Current Page

Here is the split between the "preferred" current page and the "preferred" redesign page.

“The
current homepage feels more credible and has a more professional look”

“The
current homepage feels more secure”

“The
current homepage is much easier to navigate and gather information”

“The
current homepage has a much better layout”

The results showed that users
clearly preferred the current homepage, meaning it might be time to head back
to the drawing board.

Call it stubbornness, but we
proceeded to test the redesign on our actual customers using Google website
optimizer. Google website optimizer tested the
effectiveness of our new homepage by sending half our customers to the old
homepage and half to the new homepage. From
there, website optimizer calculated conversion rate (# of transaction made /
site visitors) and revenue generated, allowing us to gauge which page lead to
more transactions.

Results of Current vs. Redesigned Homepage Test

Redesigned
Homepage Conversion Rate and
Transaction Size Improvement

Over the course of a month, our
new homepage generated 17% more revenue vs. the current homepage on similar customer
visits. 6% of the increase was due to
more converting visitors and 11% of the increase was due to people spending
more per transactions.

Our Learnings

So, why didn’t the actual
results align with the results from user testing? Sometimes, it’s hard to replicate your
customers – Regardless of the results; we are still advocates of user-testing. However, sometimes, it’s hard to find
everyday people to replicate your customers.
We’re not selling laundry detergent, so generic user panels can only
tell us so much because they don’t exactly know what your customers care about.

About the Author: Jason Thai is a marketing manager for FoxTranslate, specializing in document translation of business,
legal, immigration and academic documents in over 30 different languages.