Part 9: SEM Ad Copy Testing Insights for Your CMO

“It is the mark of an educated mind to be able to entertain a thought without accepting it.”

― Aristotle

Does DKI improve performance?

Google is rolling out the ad customizer tool, which Search Engine Land calls “Dynamic Keyword Insertion On Steroids.” So it seems appropriate to reevaluate the merits of Dynamic Keyword Insertion (DKI). There have been numerous independent studies showing that DKI lifts performance. Though many of these studies have methodology flaws (lack of statistical significance, comparisons over different date ranges, etc.), there are enough studies showing lift from DKI. Case closed, right? Not so fast. If we may make such a stuffy and high brow Aristotle reference, please “entertain the thought” that there may be more to the DKI story than meets the eye.

Boost Media Study on DKI

Boost Media’s expert team of data scientists analyzed the DKI performance of 15 premium retail AdWords accounts comparing ads containing DKI verses ads not containing DKI within the same ad groups over the same date range when matched with the same set of keywords, bids and landing pages. The findings might surprise you. But before we share these DKI insights, we must revisit the Aristotelian concept of logical fallacies.

Aristotle and Hot British Accents

American women love guys with hot British accents. Therefore, as a British guy I can have any American woman I want.

There may be a correlation between accents and high date ask-to-accept conversion rates (should we coin that the DCVR?). But it is not necessarily true that each individual woman will trip and fall head-over-heals for a guy just because he puts her bag in the “boot” instead of the “trunk.” This a logical fallacy of composition originally classified by Aristotle. It basically means that what is true of the whole is not necessarily true of each part.

When to use DKI (And how to not embarrass yourself)

Ancient Greek philosopher references aside, failing to understand the fallacy of composition can lead to drinks splashed in your face (if you are a presumptuous British guy), or poor performance (if you are a search engine marketer making a decision about how to use DKI). When we look at the data as a whole, we see that DKI drives a 14% CTR lift, but a significant 32% drop in CPI (Conversions-per-impression). Boost drilled deep into the data to learn that in high volume ad groups, this lower conversion rate finding holds true. But in low volume, long tail ad groups, DKI drives a significant CPI lift of up to 28%! Thank you Aristotle: What is true of DKI usage as a whole is not true of all the long tail parts.

But why does DKI work better in long tail ad groups?

Optimizing long tail SEM ads is time consuming and the ROI on improving any given ad is pretty low. There is not enough data associated with each ad to make statistically significant conclusions within a reasonable amount of time. Historically, the lack of data results in inaction in long tail creative optimization. Because of this, ads in a long tail ad group may generally be of lower quality. Thus, using DKI is better than doing nothing. Meanwhile, marketers more frequently optimize ads in high volume, high impact ad groups. So using DKI can be risky because these ads are already pretty well optimized. In case you are concerned about your long tail ad creative performance, we will insert a shameless plug here: Boost uses a long tail testing tool which “stacks” low volume ad groups enabling us to gain significant results by aggregating low volume ad testing. Call us if you want to know more!

Conclusion: CMO insights

Hire data-literate employees and vendors who are willing to dig in and take a second-look. Data can tell different stories depending on how it is used.

Remember what Aristotle has told us: what is true of the whole isn’t always true of each part. What other aspects of your advertising may be suffering from logically fallacious thinking?

Methodology: long tail verses aggregate performance

There are two main ways to analyze large swaths of SEM ad creative data: impression-weighted—regarding all aggregated clicks and conversions relative to all aggregated impressions. Or, an average of averages basis—evaluating the average of average ad group CTR and conversion rate. Before we bore you with mathematical detail (oops, too late!), the key difference to understand is that one method of evaluating the metrics allows the high volume ad groups and head terms to skew the results and this is how marketers normally look at data. The other way causes low volume long-tail ad groups to be more heavily weighted in the results.