Knowledge Base

A/B Testing Wizard

This wizard makes it easy to conduct evidence-based marketing by creating different testing groups, setting their priority, and also using a control group if desired.

Add actions to each testing group to create different versions of your personalized website, to be tested within the same audience segment defined by the campaign.

Set the rotation between versions to be randomly selected at each page refresh, at each session, or only one version per user.

Set control groups for even more precise analytics.

A/B Testing Steps

Target

Targeting is how you define the audience segment within which you are testing various content versions. Only visitors who match the rules you enter here will see any of the testing content, and be a part of this test.

If you want to test these versions among all of your users, you can leave the targeting portion blank.

Content

Here is where you create the various actions which will be tested within the testing groups. Add content you want to be seen by all visitors in the targeted segment (and therefore not tested) to the “Always shown” category, and all of the content you want to test different versions of into the rotation groups (test groups).

Rotation Types:

Users: Each user only sees one testing version, always

Visits: Only one testing version is shown per session

Random: Version is selected at each page refresh

Control Users: A portion of your targeted segment does not see the testing content at all

Control Visits: A portion of your visitors’ sessions do not see the testing content

QA

Reviewing A/B Test Results

In the “Performance” section of any given campaign, where you view the campaign’s overall statistics, you have a tab at the top called “Compare Actions”. Click this tab to compare your A/B testing groups.

If A/B testing is turned on, then the action statistics will be automatically grouped into your testing groups, as well as a control group if you have it activated, and the site average for overall actions. You can also select or deselect the various groups by clicking them, to include or exclude them from the comparison.

Below, you can see more detailed statistics in a table format, to compare the performance of the testing groups.

If you have specific goals tracking set up, then you can compare the testing groups’ performance on that specific goal, such as conversion rate.