Serving up my thoughts since 2004

Page 4 of 8

Hi, I run a website where I sell 7 different products each at a different price level.
I am wondering whether there’s a way so that I can track the changes I make in terms of their effect on my overall revenue?

From a marketing perspective, this raises an interesting question.
Will the test version with the best conversion rate always bring the most revenue?

If what you are testing has no relationship to what you are selling, then probably yes.

On the other hand, lets say you want to test what product you should be highlighting.
In such a case, measuring overall revenue makes sense.

One of the limitations of Google Website Optimizer is that you have no control over which combination will be displayed to a user.

Wouldn’t it be great to have total control over which variation is displayed in each section?

For example, lets say you want to test a new navigation scheme that required testing of the page header and footer, but they had to be in sync with each other. In other words, header #1 must be displayed with footer#1, header #2 must be displayed with footer#2 and so forth.

This “hack” gives you the ability to read and write which variation is displayed in each section.

Here’s a short list of some of the online marketing books I’ve read over the past few years (in no particular order).

Influence: The Psychology of Persuasion
You need to read this book. Even if your not in marketing, this is one of the better books I have ever read that explains the why and not just the what of human behaviour, specifically, in terms persuading someone to do something.

I’m trying to set up an GWO A/B test, but I’m having a problem
completing the set-up of the test because the confirmation page
requires a login. It’s not a public page because there’s sensitive
order info on there. So we’ve got the CWO code posted on the
confirmation page but GWO can’t validate it because it’s not
a public page.

However, when a real order comes through, the Javascript will
fire and it should work okay. Any idea how I can complete the set-up
of the test without going through successful validation? Right now I’m
stuck because I can’t hit the continue button in GWO to get past the
“Validate pages” step.

Jonathan

So I checked and indeed Jonathan is right.
You can’t continue without a “live” conversion page that is publicly accessible.

The solutionCreate a dummy conversion page that is publicly accessible. Enter the dummy page URL during the experiment setup and simply copy the conversion page javascript code to both the dummy page and the real page.

Once an experiment is live, Google Website Optimizer doesn’t care what the URL of the conversion page is. The conversion event will be triggered when the javascript code is executed, regardless of the page URL (if it’s not on the same domain – you need to add some code).

When creating a Multivariate experiment, this is a non-issue because you can upload the conversion page from your local file-system, bypassing the need for the file to be publicly accessible.

After a couple of weeks of testing, GWO was reporting that the new version was performing 7.03% worse than the old version. I was a bit surprised because of the radical difference between the two pages.

At that point I started looking at numbers GA provided as well. At first I didn’t see anything out of the ordinary. Then I took a look at the page performance segmented by visitor type (New / Return visitors) and my eyes opened wide.

For new visitors, the new version of the page was performing much better (over 40% improved conversion rate) and for return visitors the old version of the page was performing much better (over 20% better conversion rate).

Going forward, we now show new users the new version of the page and return users the old version. I have a few theories on why we’re seeing this behavior, but that’s a bit beyond the scope of this post. I’d love to hear your ideas in the comments.

What this shows us is that while GWO is an excellent tool, it can be a bit blunt when used on it’s own. Very often the most valuable information is in the details.

While GWO’s was originally created to test specific elements within a page, sometimes you want to test totally different versions of a page.

Lets say you want to test three totally different versions of page1.html. You create three new pages: page1a.html, page1b.html and page1c.html. You and then add some custom code to page1.html which redirects visitors to one of the new pages.

A few weeks ago Shawn Purtell from ROI Revolution wrote an excellent article on combining GWO with GA. If you haven’t read his article – please do so now. This posting describes a bug fix for his code, but you really should first read his article to fully understand how to use it.

I just found a bug in his code and am providing a fix for it. Under some circumstances, his code will provide the wrong combination number for the combination that is being displayed. Different combinations can be given the same combination number.

A few weeks ago I wrote in my Google Optimizer wish list the ability combine Google Analytics and Website Optimizer. I was actually in the process of creating a solution myself when it was brought to my attention that someone just created a solution:

I cannot emphasize strongly enough how much this adds to GWO.
Almost every metric available in GA can now be broken down based on what combination the user saw.

His solution uses the combination as a query parameter. I was originally thinking of using the user defined visitor segmentation for combinations, but after thinking about it, his solution seems better. The only situation I can think of where using user defined segmentation would be better is if you are doing advanced stuff like multi-page tests or full page testing (yes this is possible – more info to come soon).

One small note – when using the solution from ROI Revolution, be sure to call the new GA code after the GWO code. Otherwise you won’t be recording first time visitors to the page.