Split test email campaigns

Split testing your emails allows you to determine what type of emails or content work best for your audience. How people respond to your emails might differ based on what type of communication you send out and might not be consistent with responses that other organizations get. It is important therefore to test your emails to optimize your communication and make it work for your specific audience.

You can split test basing on a variety of content, for example:

Different subject lines

Length of emails

Different messaging

Different styling

Use of images

Personalization level

The idea behind a split test is to send two or more versions of the same email to a small sample of your intended audience to determine which email performs better and achieves better open, click through, and completion rates.

For example, you can send two different emails to 5% of the total list each and check the results before sending out the most effective email to the entire recipients list.

Creating a split test email

A Split Test campaign is created by going to Email > Create Email, and choosing Split Test Campaign. The email is built in the same way as a standard campaign (see Building Emails). However, you will see that you are on the first tab of the campaign (labelled "A") representing that version of the email. Also the Name of your Email is appended with "A"

Once you create and save your first version "A", you will see "+ Add another" next to the tab. If you click this, a new email message "B" is created, copying the settings from the previous version. You could then load in the content from version A via the "Duplicate from an existing email" drop down.

You can then edit version B to make any changes you want to test, e.g. the subject line.

You can add further versions if you wish, and you can have up to 8 versions.

Before you send out a split test

Once you create and save your split test, you should test them independently to make sure the formatting and content are correct. You can do this in the normal way via the test button (see this page for more information).

Sending a split test

After your 'Split Test' emails have been created and saved, you can proceed to the query builder to create your send group. Once the query has run, you will need to assign a percentage of the query audience to be tested.

The number entered will be divided among the emails you have created evenly.

Note: To do a 50/50 test you will need to enter 100 in the percentage box.

You will be able to send a test immediately or schedule the test for a specific date and time.

Reviewing the results

Once you run a split test, you will need to wait at least one hour for the software to collect the results and then you can decide which email template to send out to the remaining supporters on your recipient's list.

To view the results of your test, go to 'Email service' > 'Manage Email Campaigns' and click on the name of the specific test on the list displayed on the page. Click the yellow icon to view the statistics.

You will be able to see the list of emails you sent out as part of your test and the individual reporting (open, click through rates) for each email.

Once you decide which email performed better, simply click the 'Send as Winner' button to the right of that email .

The emails will be sent to the initially selected audience, except for users who received the test.

FAQs

Are split tests random? How do they work?

A split test sends to a randomised selection of supporters from the send group. It then sends in order, so Email A is sent, then Email B, and then A and B again.

When you send the winner, the selection list you send a split test to will not requery - i.e. the original selection list chosen at the send, or scheduled send, is used for the entire process.

Can I automate the winner?

Yes, you can automate the winner selection based on criteria you define. For more information on enabling and setting this up, check the page about the email settings.

How do I use split tests in my query?

Note that the Name of Email field is what is listed in the query builder under the Email criteria, and not the name of the email campaign, i.e. the name that is appended with A or B.

If the winner was version B, then the original split test and remaining supporters would be included under that email name.