A/B Testing in Google Analytics with Event Tracking

Rostislav Mykhajliw
Founder of TrueSocialMetrics.com
~ 6 min

As you know, by default, Google Analytics experiments feature works only with separate pages and
it works well when you are starting advertising campaign. So you can control destination url.
But what to do, if you don't control your traffic sources: search, referral, social media postings.
That's the main issue. But we have a simple solution for it.
The solution requires:

Let's start testing

/ index page has an image on it by default, but we want to replace it with video for 50% visitors and measure sign up rate for each case.
I've created a simple js for A/B testing with saving allocation in cookies.

I prefered html5 code (but if you want you can always step back and use legacy flash object).
Our logic is simple: if the cell name equals "Video" update element id "ab-testing-ImageVsVideo" by given player html code.

Everything works well, but we haven't tracked our goals. How do we know which test variation wins?
Google Analytics provides great feature called Event Tracking. It's mostly used for tracking user action on the page like button press or some ajax action tracking.
But we will use it for tracking our test's cells allocation.
There are 2 required arguments: Category and Action, in our case "AB-Testing" category for all a/b tests
and action "ImageVsVideo-Image" (or ImageVsVideo-Video) as identificator.
Working script for our page.

You can find sample at / source code, I also have put allocation function into the file ab-testing.js for easy usage.

Google Analytics reports is the next step

Our original goal was to increase "SignUp rate", but also we wanted to know an
economic outcome, general conversion rate and goal completions.
So we selected following metrics:

Unique visitors

Sign Up (Goal11 Completions)

Sign Up (Goal11 Conversion Rate)

Per visit goal value

Goal Convertion Rate

Goal Completions

Also it's nice to split "New" and "Returning" visitors, because mostly when we're talking about "SignUp" we care about new users.

So let's go to custom report and create a new "A/B testing"
It's great! But we still have a full list of visits without the difference "Image" or "Video", so we have to create 2 custom segments.
Go to "Advanced Segments" and create "New Custom Segment" - "Image".
Find metric "Event Action" contains "ImageVsVideo-Image".
Repeat the same for "Video" segment with "Event Action" equals "ImageVsVideo-Video".

Finally, apply both segments "Image" and "Video" and you will see something like this:

As a result: Video conversion rate 13% vs 10% with Image, higher economic value, but Goal Conversion Rate is a bit lower.
So we need more statistics. Let's select secondary dimension "Traffic type".
For direct traffic results are almost the same, but for search traffic "SignUp rate" and Economic value are much higher.
But from another perspective, for existing users conversion is lowering. It shows that for people who already visited our website
"quick response" is more important.
So it makes sense to show "Video" to the new clients who come from search engines.

Update from 05/07/2013

I've updated "allocation code" by adding the multi-tests support for the same page. For example, on a current blog page - 2 tests:

There are 4 different "singup" forms, with a different messages (but the rest of the 3 are hidden). If you open page in a source mode you will find them.

The second test is a message under the form with a link to "How-it-works" or "Singup" pages.

The full code is here, also you can download our latest version (remember we're using specialized async js loader, if you don't use it, you have to replace head.ready(function(){ for $(document).ready(function(){ ).

The main difference is that new code doesn't require to write any piece of code on JS. You only need to add the class "ab-testing" to the any element you want to test and specify the testname and the cellname e.g. <div class="ab-testing" data-ab-testing-test="signupblogtext" data-ab-testing-cell="how-it-works">some code for testing</div>.

Let's look on a real sample of applying this testing from the scratch.

As a result, 50% of users will see one block with "how-it-works" link and another 50% - the second one.

Testing

My code is storing allocation in users cookies, which means allocation is happening only once, so if you want to test it you need a clean browser without cookies. But much a simplier solution is using Google Chrome feature Incognito Window. It provides absolutelly clear new browser window without any cookies. As a result, you can open the page several times and check if everything works properly.