Establishing the real value of display through placebo-based attribution

The progress in ad tech in recent years goes hand in hand with increasing concerns from advertisers around viewability, fraud, measurement, brand reputation and industry malpractices. In October 2017 P&G, the world’s biggest advertiser, cut back its digital advertising budget by 200 million dollar1. In February Unilever, the world’s second largest advertiser, announced it may also be cutting digital ad spend2. The writing has been on the wall for a while for display advertising:

Average display advertising click-through rate in Europe is at 0,3% or lower 3

Display advertising viewability in most European countries is around 50%-60% 4

20% of internet users in Western Europe use an ad blocker 5

With the above statistics in mind, how can you be sure as an advertiser that your investment in display advertising actually makes a difference to your bottom line?

Awareness or conversion?

Many advertisers have no sound model in place to show the real correlation between display advertising and conversions. Conversion revenue is attributed on a last-click basis and because of poor CTR’s the revenue usually is far below the ad spend (read on for post-view conversion). For this reason marketers often turn to awareness as an objective to justify digital ad spend. But how do you measure that your advertising campaign has generated awareness?

The most accurate way to measure awareness is through upper-funnel metrics such as brand awareness, ad recall, brand favorability etc. which are measured through post campaign testing that requires interviewing human beings. Since this is very expensive most advertising campaigns, even in large companies, are not post tested. Instead many marketers and their media agencies justify ad spend through fuzzy metrics such as impressions and post-view metrics.

Impressions are not views

An impression means an ad was served. It doesn’t mean the ad was actually viewed. The IAB standard for ad viewability stipulates that a valid ad view is generated when 50% of the pixels of the ad were in a viewable position for at least one second. Is it really possible to understand what an ad is about after seeing half of it for 1 second ? I beg to differ. Even so, with this highly lax definition viewability rates are around 55% for many advertisers. Meaning that as an advertiser you may reasonably assume that almost half your budget goes down the drain before you even started.

“Half the money I spend on advertising is wasted; the trouble is, I don’t know which half.” – John Wanamaker (1838-1922)

Building on that assumption: if the other half of your ads are served in a viewable position, this still does not mean they were actually seen. Heatmap studies have confirmed that website users tend to ignore display ads.

What about post-view metrics?

Post-view conversion refers to website users who did not click on display ads but who were exposed to them, then visited the advertiser’s website through another channel than display ads, and then converted (buy something, fill out a form etc.). So the outcome is attributed to the ad impression on the assumption that viewing the ad made a difference. And this is where post-view attribution breaks down. Because, once again, ad impressions don’t mean ads were viewed. In fact, about 50% of those ads weren’t viewed.

How then can we measure the true impact of display ad campaigns in terms of conversion attribution? Enter placebo testing.

Placebo testing

The concept of placebo testing is rather simple: you ramp up two identical display advertising campaigns with the same targeting and landing page. The only is difference is the ad creative. One ad communicates your brand message, the other ad, also called the placebo ad, communicates something that is totally alien to your brand. If you are in fashion this could be dog food, travel insurance, sanitary towels or whatever really. You split run the campaign and then you compare the post-view conversion stats. If your dog food ad generates a significant amount of fashion purchases (here’s a little secret: they always do) then you now know that your post-view conversions are to be taken with a margin of error. How big a margin of error? The calculation as seen in the example below is rather simple:

Creative A: brand message => 100 post-view conversions

Creative B: placebo ad => 50 post-view conversions

Margin of error = 50/100 x 100% = 50% => only 50 of the 100 post-viewed conversions that were attributed to creative A are genuine.

Nice theory but…

There’s no but. This is the way to approach it. The outcome for one of our clients was that post-view conversions that for years had been attributed to their display advertising campaigns had to be discounted by almost 40%. There was a deep-rooted belief with this client that display ads were indirectly responsible for the bulk of their online revenue, based on years of media agencies reporting shiny post-view metrics.

Their marketing budgets were spent accordingly with a heavy emphasis on display advertising. Yet when all display advertising was paused over a brand safety concern their website traffic wasn’t impacted at all. People barely click on display ads remember. More importantly their online revenue was not impacted either in the many months that followed. Food for thought for next year’s marketing budget allocation.

What about programmatic?

Programmatic advertising is still display advertising just done in a different way. Placebo testing on programmatic campaigns is not possible though. Since you buy impressions one at a time it is impossible to do split run testing. But you can identify on which sites your programmatically bought ads frequently appear and then do a media buy on one of those top sites to run your placebo test. Chances are you’ll be very surprised by the outcome. This doesn’t mimic programmatic buying, but it’s pretty indicative.

What about remarketing?

Time and again I hear clients stating how well their remarketing campaigns work. Of course do people who already visited your website convert at a higher rate. But do you really believe that showing that one additional display ad pushed people over the edge towards that conversion? I encourage you to challenge this common belief using objective data from placebo testing.

Most of the remarketing I come across is quite poor. If I have to single out one thing that particularly disturbs me then it is remarketing ads that are shown to me after I have already converted. This regularly happens to me when I buy travel or make purchases on e-commerce websites. It skews remarketing effectiveness data and is a sheer waste of your marketing money.

So think critically about your display advertising. Don’t let yourself be led by agencies who earn their money from media buying. Ask the difficult questions and use objective neutral data to answer them. If you’re curious about placebo testing or in need of help with your attribution questions, don’t hesitate to give us a call.

Web analytics tools heavily overstate the number of unique visitors your website has. The degree of error ranges anywhere between a few dozen percent to a few hundred percent. Here we discuss the variety of causes for this erroneous unique visitor count.

Javascript disabled

The smallest issue for distorting unique visitor count is visitors who have Javascript disabled. For many years the percentage of users who have Javascript disabled has evolved around 2%. Often these are visitors who use Firefox NoScript add-on. When users who have Javascript disabled visit your site none of their activity will be measured. This can be compensated somewhat by adding a tag to your normal analytics tag.

In the Google Analytics example below this will register a hit via the Measurement Protocol even when Javascript is disabled. This will enable you to register a pageview but most of the other information that a tag would register won’t be captured.

Ad-blockers

The use of ad-blockers, particularly by younger users, significantly increased in recent years. In many European countries ad-block usage is above 20%. Many of these ad-blockers have the ability to block web analytics trackers. Some ad-blockers block analytics tracking by default, others make it very easy to enable it.

The margin of error this creates varies according to your user base. One experiment established that Google Analytics script was blocked in 11% of cases.

In-private browsing

Nearly one out of five internet users use in-private browsing. With regards to analytics tracking this means only temporary session cookies and no permanent cookies to identify unique visitors will be placed. When visitors come back to your site they will be considered a new visitor thus inflating your unique visitor count. However, their session information and hits will be registered correctly.

The above applies to nearly all browsers, except Firefox. In-private browsing in Firefox means the NoScript add-on will block any form of Javascript tracking. As such no measurement will be made at all.

Cookie deletion

It is hard to find statistics on cookie deletion behaviour. Yet when I ask in the many analytics courses that I teach whether participants delete cookies I find that without exception several of them do at varying frequencies (anywhere between 6 weeks to several months). If I were to stick a number on it I’d probably pick 20%. Visitors who delete cookies obviously heavily distort your unique visitor count since they will be counted as a new visitor every time they come back to your site after having deleted their cookies.

Multi-device & multi-browser usage

Users who use more than one device to visit your site will have a tracking cookie on each
device and will therefore be counted as multiple unique visitors. The question is, how
many of your visitors do use multiple devices. The only way to prove this is by making visitors login to your site every time they visit. You will then be able to record a unique user id instead of a cookie id.

In the example on the right taken from a site where users are always logged in we see that about 30% uses more than one device to visit the site. If no unique user id would have been captured this would mean the unique visitor count would be inflated by 30%. On your site the same thing happens although maybe to a lesser degree (the example was taken from a site to which visitors return very frequently).

When users use different browsers on the same device to visit your site the same principle applies. They will have a cookie in each browser will leads to inflation of the unique visitor count.

In conclusion

There are different reasons why the unique visitor count of your analytics tools is overstated. Some issues cause that unique visitors are underreported (javascript disabled, ad-blocking), other issues will inflate the unique visitor count on your site (in-private
browsing, cookie deletion, multi-device usage, multi-browser usage).

Clearly inflation is far more likely to happen than under-reporting. Yet, it is virtually impossible to quote a generic inflation factor.

In the example on the right from a site where users a always logged in we confronted the number of unique user id’s with the number of unique cookies in Google Analytics. The result: 93.681 unique user id’s matched to 312.577 users or unique cookies.

The real number of unique visitors was overstated by more than a factor 3 ! With some visitors equaling as many as dozens to hundreds of unique cookies !!