Using A/B testing to boost your newsletter results

Published on February 17, 2005.

Most Popular

By Karen J. Bannan

A/B testing—comparing the results of two slightly different advertising campaigns—is a common practice in the offline world. It should be just as important in the online world, according to experts. Dave Lewis, VP-market development for Redwood Shores, Calif.-based e-mail infrastructure company StrongMail Systems, explained how marketers can use A/B testing online.

1) Test and test again. It might seem like a lot of work, but marketers should conduct some testing with every campaign or newsletter they send out, Lewis said. "You never reach a point where your campaign is perfect, because the technology and strategy you use will become passé," he said. "Look at positive and negative response, delivery rates and the types of failures you’re seeing."

2) Take it offline. Integration is a big buzzword this year, and for good reason. You can use an online campaign to test copy or graphics that you plan to use in a print or direct mail campaign. "No other medium out there lets you get results within hours or days," he said.

3) Give yourself a little space. Don’t make the mistake of testing more than one campaign at a time, especially if you don’t use a campaign management tool. "You don’t want to outsmart yourself and not know which results are attributable to which test," Lewis said.

4) Ask for feedback. While you should always give your customers a way to communicate with you, go one step further in your test e-mails by asking for feedback, Lewis said. "Once you get that feedback, do something about it immediately," he said.

5) Remember: A/B isn’t just for content. A/B testing is crucial when you purchase a new list, Lewis explained. Use it to look at open and click-through rates. "Test a portion of the new list source against a portion of the competing list source to see if they perform at the same level and produce a higher ROI," he said. "You can’t just be looking at the positive response, you must look at the negative response as well. If you get complaints or lots of opt-outs, it could cause the rest of your campaign to be jeopardized."