Multi-channel advertisers often ask me how to measure the effect of their TV ads on their online marketing channels. The question is important because it is the first step in the answer to the media mix question: What is the right distribution of budget across TV and online channels to maximize return on investment.

There is typically a halo effect in TV campaigns, e.g. when you run a TV campaign you typically see its effect last on search for a duration longer than the TV campaign.

If the online and offline marketing teams are separate (they typically are) then the debate can be fractious.

While the econometric modeling is a vast subject, I want to present a simple method to help you answer the basic questions. We shall use the data of a direct-response advertiser that uses TV to drive people online and finish the sign up process there.

Simply put, econometric models aim to quantify the effect of several variables into a metric of interest. The basic assumption of econometric models are that the variance (fluctuations) seen in the output are due to changes in certain input variables.

In our case, we shall assume that the controlling variables for the number of brand impressions on search are a linear function of search spend and TV spend.

Since non-brand searches are influenced by many factors (not least the search engines matching algorithms), we shall only consider branded keyword metrics in this exposition.

Further, since SEM spend on brand terms is almost constant (all keywords here were bid to position 1 for the duration of the campaign),the baseline search impressions (search impressions without TV ads) can be expressed as a function of SEM spend:

Since SEM spend is constant, we can then define the impression jump as:

This simplifies to:

This is a simple one variable regression and can be plotted as shown:

The graph shows a strong connection between search impressions and TV spend. For instance: it shows that when TV spend is $600,000, then the impressions in search jump by 70%.

At $1.4 million in TV spend, the impression volume jumps by 200%. The R^2 of 0.86 indicates that 86% of the change in branded search impressions in this example are explained by changes in TV spend.

Limitations:

This is a very simplified econometric model that does not account for time lag factors. If you are interested in more sophisticated time series models I would suggest looking into methods such as Holt-Winters, ARMA, ARIMA models.

I have chosen only two factors: TV and Search spend to explain the fluctuation in search impressions. There might be many other factors that I have not accounted for to build my model. An econometric model for a large brand typically includes several macro-economic indicators as well as other marketing channels such as PR and print. Variable selection for these advertisers is an art and science in itself. As a result, while the fit of my model is reasonably good, my prediction accuracy over longer periods with the model might be poor if a yet unaccounted for factor becomes meaningful.

You might wonder why I chose search impressions and not search conversions as my metric of interest. The reason is I typically find impressions regress much better than conversions when looking at the offline –online conversion. To connect conversions to offline spend, I would need to build a more robust model.

This model has a basic problem in that it assumes that the properties of data fed into it have the same properties regardless of time. In other words, it assumes that the effect TV and search spend will have on search impressions is the same if the day is a prior month’s or a prior years. It also does not account for seasonal factors. While this assumption works when analyzing data from a stable period, it is clearly not true when markets change due to macro economic factors as well as due to seasonality. A sophisticated way to over come this is to use time series model , where time is a factor in the models.

I hope my exposition helps you get started in analyzing the offline-online effect or other places where detailed granular data is not available. While not the most accurate or the most comprehensive model, I have had success good success with this simple method in connecting the dots between the online and offline world when the dataset is from a stable time period. I hope you find this simple model useful, too.

Some opinions expressed in this article may be those of a guest author and not necessarily Search Engine Land. Staff authors are listed here.

Sponsored

http://www.rimmkaufman.com/ George Michie

Terrific piece, Sid. I was about to raise the time duration issue (lifts impressions by X% for how long?) but then you raised it yourself in #4.

Everyone in our space is interested in proving the offline impact of online ad buying, but too few folks are willing to acknowledge much less study the impact of offline marketing online. Kudos for doing so!

http://twitter.com/serbayarda Serbay Arda Ayzit

This is really nice topic, do you count only ppc part in your model? And you didn’t take search clicks to the web site, is there any reason not to take it?
One more thing, i think we must take organic searches in this model too but not provided issue will destroy data quality am i right?

Armen Avedissian

Excellent article indeed, this is a good starting point, however there are so many other areas to take into consideration, would enjoy discussing more in detail.

At Convertro we specialize in exactly these areas and measure the impact of offline on other offline channels as well as all online channels.

A great place to start & I’m glad you were forthright with the limitations of such a simple model. I briefly modelled media mix in one of my previous positions, and we used GRPs rather than ‘spend’ on advertising as our independent var from the TV channel. Since GRPs can vary in price depending on the sub-channel, it may be more advantageous in such a simple case to model reach vs. impressions rather than spend vs. impressions – you may get a more consistent, and actionable result if you invest in more than one TV sub-channel.

I am also glad you brought up the lag issue, as it’s definitely a big one when trying to model any marketing mix, especially any type of online-offline attribution. Remember, the lag will also depend on the type of advertising and the channel itself. TV may leave a longer lasting impression than radio, or a branded special promotion might have more staying power to fuel searches than a typical, run-of-the-mill awareness advertisement. That being said, I would caution all but the most learned and experienced in econometrics from attempting ARMA or ARIMA modeling. A simple AR model – even for media mix – should do the trick in a simpler way without much risk of leaving out any important indicators. You can use the partial autocorrelation function to help determine your starting lag lengths.

http://www.facebook.com/people/Sid-Shah/100002282332518 Sid Shah

George: Thanks for the compliments. I had a tough time writing this piece because I wanted to keep it simple but not over simply it.

Serbay: Yes, you can measure the impact on total clicks too. The dependant variable will change and I am sure you will see organic clicks jump too. For this dataset I did not have organic or direct load data.

Kenny: You bring up a good point about GRP. I have done simple modeling of GRP vs SEM clicks and TV spend vs clicks and found GRP to have a better connection to SEM clicks.

I am curious to know why you caution against ARMA models. Could you explain ?
Thanks
Sid

It’s my personal process to try and use the simplest form of modeling I can to achieve a certain goal, while still preserving data and statistical integrity. When you move from the simpler models, like ols, gls and ar, to the more complex models, such as arma, arima, arch, mle, or others etc. There are many more ‘quirks’ and ‘rules’ the data must follow for the results of these more complex models to have statistical validity and be actionable. For instance, in ARIMA modeling with time series, you have to start concerning yourself with random walk, or whether or not the model should contain a constant, among other things.

Even having taken 4 graduate econometrics courses myself, I find I am hesitant to explore these more complicated models in an applied setting. Among other things, I have time constraints to meet. If an ARIMA model would be 5% more accurate, but take 40% more time to construct, is that an acceptable margin of error for the project I’m working on? Most of the time – yes.

As for why others should possibly avoid the more complicated models, most people who have taken a statistics course are familiar enough with OLS to know about heterskedasticity and the other ills associated with linear regression – at least enough to where their results will be, in general, actionable and statistically valid. It’s kind of hard to really mess up OLS. But with the more complex models, they’re likely teaching themselves along the way, or possibly just plugging numbers in to MiniTab and hitting ‘go’ without any proper knowledge of whether or not their results even mean anything!

I’m certainly not trying to steer people away from learning these more complex statistical models. But it does take time. They certainly have their place, but I find it lies more with the academic world than the (typical) fast-paced business world.

This is a thrilling discussion, to be sure, and much kudos for bringing the more advanced side of econometrics to light. There is a much, much deeper hole than OLS, afterall :)

Cheers,
Kenny

PeckLorrie254

Here are few tips by using which you can earn so much money in couple of days. In start you will face so many difficulties but later on you will get various ways to earn money over Internet. Everything takes some time to get to know, same the case with it, but we will tell you the kick-start way of earning through Internet which would be much more worthwhile for you. Last week I got a site, where you can kick a good start, Look this for further details.  http://earnusdathome.weebly.com/

http://www.facebook.com/people/Sid-Shah/100002282332518 Sid Shah

Kenny,
I quite agree with your philosophy. I like to keep it as simple as possible if it gets me far enough to the answer. There is an opportunity cost of time in business !

As far as AR/MA/ARMA models.. I have always had a bit of a philosophical issue with them. I like deterministic models to explain trends and only use time series models when I dont know the mechanics of the underlying process.

Attend Our Conferences

Attend Marketing Land's SocialPro conference and learn fresh new strategies and tactics from some of the savviest brands and digital marketing agencies managing earned, owned and paid social media marketing campaigns across multiple platforms. Visit the SocialPro site to learn more..