Awesome A/B Tests for Newsletters

A/B tests are one of the most valuable tools in a data-driven marketer’s toolbox. Not only do they help you optimize everything from emails to landing pages, they also teach you a lot about your business’s unique audience.

However, you shouldn’t run A/B tests just for the sake of them. Before you start, have a clear view of what you are testing for — for example, you might want to test out a new design before completely adopting it.

When it comes to email, there are three main ways you can measure the success of your A/B tests:

Open Rate — out of all the emails delivered, how many were opened

Click Rate — out of all the emails delivered, how many were clicked

Conversions — out of all the emails delivered, how many resulted in a purchase (or other action that you have set as your conversion metric)

And to optimize these, there are five main components of an email that you can test:

Subject Lines

Preview Text

From Address

Content

Layout

You should never simultaneously test these — it’s important to have a control group so you can tie any changes in opens, clicks, or conversions to only one variable.

When you A/B test a newsletter in Klaviyo, the test will run for a period of time on a sample percent of the entire list. You can choose the duration of the test and the sample size. In most cases, you should keep your testing period at two hours or less to minimize the impact of timing on the A/B test.

Before you start A/B testing your newsletters, it’s a good idea to have a broader concept that you’d like to test rather than something that is specific to the email you are running the test on. While A/B testing newsletters ensures that the winner goes to the majority of recipients, to truly learn something about your audience, you should run a series of tests over a longer span of time.

For example, you may want to test how using emojis in your subject lines impacts open rates. Rather than draw conclusions from the results of a single test, you should continue running similar A/B tests for at least ten campaigns. Based on what you find, you can make an assessment of the impact of emojis (or find that you want to run the test on a few more campaigns).

Emojis aren’t for everyone. They convey a less serious tone, so you’ll want to make sure the products you sell fit with this tone. Is your audience made up primarily of millennials? Give emojis a shot. Do you market mainly to surgeons? Maybe skip them.

A sense of urgency

Using a sense of urgency in your subject line — like “Only 1 day left to preorder our newest shade of lipgloss” — can boost open rates. Test them to see how your audience responds.

At some point, you’ve probably received an email with a subject line like “Have you heard?” Teasers (sometimes not-so-affectionately called clickbait) can tempt recipients to open your email. But it’s important to look at this in relation to your click rate, too. A little mystery is good, but you don’t want to mislead your subscribers.

Humor

Based on the products you sell, you probably already know if humor has a place in your marketing strategy. However, it’s a good idea to test humorous subject lines to see how your audience responds before completely adopting them — after all, you don’t want to find out the hard way that you’re not as funny as you think you are. Preview text, also called preheader text, is displayed after the subject line in the inbox preview — it is pulled from the content at the very top of the email. When testing your preview text, measure it against your open rate, even though it is reflected in the content of the email. Because this is what is displayed in the inbox preview, it will have more of an influence on open rates than click rates.

You can test many of the factors mentioned above under Subject Lines. Another fun idea is to test an interaction between your subject line and your preview text like the one below:

Your from address should be clear and indicate who is sending the email. That said, you can try sending your email from a company versus an individual. Here are some ideas how:

From Address

Subject Line

Klaviyo

Check out our latest blog post

Marissa Petteruti

Check out our latest blog post

Marissa | Klaviyo

Check out our latest blog post

The Klaviyo Marketing Team

Check out our latest blog post

Marketing at Klaviyo

Check out our latest blog post

LayoutExperimenting with the design of your newsletter is a great way to optimize your click and conversion rates. For example, your current newsletter might have a great click rate because you display many links, but your conversion rate might be low. You could try using hyper-focused CTAs to make your click and conversion rates more aligned.

Design is great to A/B test because it gives you a lot of flexibility. In your newsletters, try playing with the balance between text and images, your banner images, footers, etc. Because you’re testing the design as a whole, it’s OK to make multiple changes to the layout before testing it against your original template. Just make sure the other components of the email — the copy, subject lines, etc. — are consistent. Bottom LineA/B testing your newsletters can help you discover the best way to effectively communicate with your unique audience. Because all businesses are different, there is no silver bullet for subject lines, from addresses, or layout that works for everyone — so, to optimize your newsletters, you should to test to see what works best for you.