Transcription

1 Do Display Ads Influence Search? Attribution and Dynamics in Online Advertising Pavel Kireyev Koen Pauwels Sunil Gupta Working Paper February 9, 2013 Copyright 2013 by Pavel Kireyev, Koen Pauwels, and Sunil Gupta Working papers are in draft form. This working paper is distributed for purposes of comment and discussion only. It may not be reproduced without permission of the copyright holder. Copies of working papers are available from the author.

2 Do Display Ads Influence Search? Attribution and Dynamics in Online Advertising Pavel Kireyev Koen Pauwels Sunil Gupta 1 February 9, Pavel Kireyev is a Ph.D. student and Sunil Gupta is the Edward Carter Professor of Business Administration at the Harvard Business School, and Koen Pauwels is Professor at Ozyegin University, Istanbul, Turkey.

3 Do Display Ads Influence Search? Attribution and Dynamics in Online Advertising Abstract As firms increasingly rely on online media to acquire consumers, marketing managers feel comfortable justifying higher online marketing spend by referring to online metrics such as click through rate (CTR) and cost per acquisition (CPA). However, these standard online advertising metrics are plagued with attribution problems and do not account for dynamics. These issues can easily lead firms to overspend on some actions and thus waste money, and/or underspend in others, leaving money on the table. We develop a multivariate time series model to investigate the interaction between paid search and display ads, and calibrate the model using data from a large commercial bank that uses online ads to acquire new checking account customers. We find that display ads significantly increase search conversion. Both search and display ads also exhibit significant dynamics that improve their effectiveness and ROI over time. Finally, in addition to increasing search conversion, display ad exposure also increases search clicks, thereby increasing search advertising costs. After accounting for these three effects, we find that each $1 invested in display and search leads to a return of $1.24 for display and $1.75 for search ads, which contrasts sharply with the estimated returns based on standard metrics. We use these results to show how optimal budget allocation may shift dramatically after accounting for attribution and dynamics. Although display benefits from attribution, the strong dynamic effects of search call for an increase in search advertising budget share by up to 36% in our empirical context. 1

4 Introduction Firms are motivated to spend more of their marketing budget online as consumers increasingly use online media to find information. Worldwide digital advertising spending in 2012 was $103 billion, or about 20% of total money spent on advertising, and is expected to increase to $163 billion, or 25% of total advertising spend, by the end of 2016 (emarketer 2013). In 2012, almost half of all digital ad dollars worldwide were spent on paid search, and 38% were used for display ads (ZenithOptimedia 2012). The introduction of online metrics such as click through rate (CTR) and cost per acquisition (CPA) by Google and other online advertisers has made it easy for marketing managers to justify their online ad spend in comparison to the budgets used for television and other media. However, these metrics suffer from the fundamental problem of attribution, since they give credit to the last click and ignore the impact of other ad formats that may have helped a consumer move down the conversion funnel. Consider, for example, a consumer searching online for a bank to open a new checking account. During this search, the consumer sees a paid search ad for a particular bank, clicks on it, and converts, recalling that she saw display ads of the same bank a few weeks earlier. How should search and display ads be credited for the conversion, and to what extent? Most managers recognize the attribution problem, and intuitively believe that display and search ads interact to influence consumers. Recently, analytical firms and ad agencies have started addressing this problem, but most of their solutions tend to be ad hoc. For example, some industry models give equal weight or credit to all ad exposures received by a consumer in, say, a two week period; others give more weight to recent ad exposures and exponentially lower weight to past ads (Havas Digital 2010). As firms spend more of their ad dollars on online search and display, managers and researchers alike recognize a need for more careful attribution adjustment that takes into account the journey consumers follow before conversion. Unfortunately, the effects different advertising media have on consumers at different stages of consideration are not yet well 2

5 understood (Marketing Science Institute 2012). In this research we use time series models to infer the interaction between search and display ads. Specifically, we address the following questions: Do display ads influence paid search and vice versa? If so, how large are these effects and what dynamic patterns do they follow? What are the implications for online marketing metrics and optimal budget allocation? This research draws broadly on two streams of literature online advertising effectiveness (specifically to our context, display and search) and the spillover effects of online advertising. In the context of display ads, researchers have studied the impact of ad exposure on click through behavior (Chatterjee et al. 2003), long term brand awareness (Drèze and Hussherr 2003), and repurchase decisions (Manchanda et al. 2006). Research has also explored the potential of targeted display advertising (Sherman and Deighton 2001, Shamdasani et al. 2001, Moore et al. 2005) and the consequences of its intrusiveness (Edwards et al. 2002, Goldfarb and Tucker 2011a, 2011b). Lewis and Riley (2011) use a randomized experiment to measure the causal effect of online display advertising on offline retail sales. In the context of paid search, researchers have focused on understanding optimal advertising strategy in complex search engine environments. Ghose and Yang (2009) and Rutz et al (2012) adopt a keyword specific approach to understand the performance of individual keywords and guide optimal keyword investment decisions. Further work examined spillover within search. Yang and Ghose (2010) identify complementarities across organic and paid search listings, and Rutz and Bucklin (2011) find spillover effects from generic search to branded search. Wiesel et al (2011) model consumer progression through the purchase funnel, and explain how online advertising may drive sales in the offline channel. In contrast to the above study, which mainly examines advertising effectiveness within a particular online channel, we study ad effectiveness taking into account the interaction and feedback between both search and display channels. Furthermore, we focus on the role of search and display advertising in 3

6 customer acquisition in the commercial banking industry, where consumer decision process tends to be longer and more involved, and the attribution problem is more severe. Several studies consider spillovers and synergies in online and offline consumer behavior. Naik and Peters (2009) propose a hierarchical model to capture synergies within the offline channel and across online and offline channels. Their model builds on earlier work (Naik and Raman 2003), and argues that investing in offline and online advertising simultaneously generates greater revenues than investing in each channel individually. A small number of studies examine the interaction between paid search and display. Papadimitrou et al (2012) conduct a field experiment to explore the impact of display exposure on search queries. They find that exposure to a display ad increases the number of relevant search queries submitted by 5% 25%. Lewis and Nguyen (2011) conduct a field experiment to explore the impact of display advertising on advertiser and competitor branded search queries. Using a very short time window (10 minutes) they find a 27% 45% lift in searches attributable to the display advertising exposure. A number of studies conducted by industry researchers also explore the impact of display advertising on paid search. A survey conducted by iprospect (2009) finds that about 50% of all internet users react to a display ad by conducting a search related to the brand or product described in the ad. The study finds that 14% of users make a purchase after conducting the search. A study by comscore (Fulgoni and Mom, 2008) finds a 38% lift in branded search activity for consumers exposed to a display ad. The study tracked individual consumers exposed to a display ad, and compared their behavior with a group of similar consumers not exposed to display advertising. An icrossing study (Malm and Hamman 2009) finds a 14% change in search visits after a company activated its display advertising campaign. In our application for the bank, the bank s ad agency also conducted an experiment to find that display ads improve search ad conversion by 15 20%. As these studies suggest, display ads appear to influence the effectiveness of search ads. However they lack two important elements that we consider in our study. First, almost all of these studies ignore the dynamic effects of advertising, whereby display ads may impact consumers search behavior over time. Studies that attempt to incorporate dynamics do so in 4

7 an ad hoc fashion. For example, the ad agency for our bank decided to use a two week period (an ad hoc assumption) to examine its effect. In our application, we show that these dynamic effects are very strong and may last several weeks. Ignoring them can lead to significant underestimation of the effectiveness of online ads. Second, most of the previous studies used click through rates or similar metrics to measure the impact of display ads on search. In contrast, we examine how display ads influence search clicks, conversion and ultimately the profitability of the firm. This allows us to determine appropriate budget allocation between search and display. The remainder of this article is organized as follows. First, we present our conceptual framework by explaining the relation between attribution and consumer funnel progression. Then, we present the data, modeling methodology, and empirical analysis. To conclude, we provide a set of attribution and dynamics adjusted marketing metrics, and discuss managerial implications. Conceptual Framework The consumer journey can be conceptualized as a conversion funnel. A consumer may be exposed to a brand through display ads, she may click on these ads to get more information, and may eventually convert. This is the direct impact of display ads on conversion that most studies find to be very small. Alternatively, a consumer could be actively searching for a product online, where she encounters a search ad, clicks on it, and converts. This is the direct effect of search ads, which is usually bigger than the direct effect of display ads. It is common to measure these direct effects of display and search using online metrics such as CTR, CPC, and CPA. Besides direct conversion, passive forms of advertising exposure may also influence consumers consideration sets, and subsequent active engagement with the firm moves consumers down the funnel towards conversion. In this scenario, display ads may influence consumers at the top or middle of the purchase funnel while search ads may have more impact at the bottom of the funnel. 5

8 Figure 1 shows how the firm s online advertising strategy may influence consumers purchase behavior and the firm s budget allocation. In this framework, the firm allocates a budget between search and display ads that determines the number of ad impressions to consumers. These in turn affect display or search clicks, and eventually, conversion. Two important aspects of our framework should be noted. First, we expect strong interaction between search and display ad impressions and clicks. The empirical results will show if display ads indeed influence search ad effectiveness, and if so, by how much. Second, the system explicitly recognizes endogeneity, whereby the firm s advertising budget influences consumers exposure and purchase behavior, which in turn affects how much the firm spends on advertising. Figure 1: Impact of Online Advertising on Consumer Behavior 6

9 Data Description We use data from a large commercial bank that operates mainly in the southern U.S. After the financial crisis, advertising to acquire new consumers became increasingly important given reduced margins and declining consumer confidence. The bank invests heavily in both paid search and display ads to acquire customers for its checking account. For the calendar year 2010, the bank and its advertising agency provided us weekly data on the bank s online marketing expenditure, search and display impressions and clicks, and the number of online applications completed by consumers for a new checking account. The bank invested about $1 million in online advertising, almost equally split between search and display. 2 There are two limitations of our data set. First, the bank does not track if online advertising influences consumers to open a checking account in its retail branch. This means that we cannot investigate the impact of online advertising on offline behavior and vice versa. Second, our dataset consists of only aggregate levels of consumer behavior. Although the lack of individual level data is a limitation for our study, managers routinely use aggregate data to assess the performance of their online campaigns. Paid search data capture weekly spend, clicks, impressions, and the number of applications completed through the paid search ad s landing page. The display data also contain information on weekly spend, clicks, and impressions. Using internet cookies, applications completed were attributed to display advertising if a consumer had seen a display ad at least one month before converting through the display ad network s landing page using organic search or a direct link. 3 However, the display advertising data excluded display driven paid search conversions, as the paid search campaigns are overseen by platforms maintained by search engines unrelated to the display ad networks. 2 To maintain the confidentiality of the client bank, we have disguised some of the data while maintaining the relationship between the variables of interest. 3 This is another form of ad hoc attribution between display ads and other online media. However, we do not investigate this in our study due to the lack of individual level data available to us. 7

10 The bank invested in five search engines and eleven ad networks. We aggregate over search engines and ad networks to the week level to avoid over parameterization as our primary interest lies in the interplay of search and display advertising, as opposed to the performance of individual search engines or display ad networks. Furthermore, the bank s limited investment over a number of smaller ad networks and search engines makes it difficult to estimate the impact at the level of a search engine or ad network. Table 1 presents a correlation matrix of the variables in our data. The notations used for the variables are indicated below: : Checking account applications completed through paid search in week t. : Checking account applications completed after exposure to a display ad. : Paid search ad impressions. : Paid search ad clicks. : Weekly expenditure on paid search advertising. : Display ad impressions. : Display ad clicks. : Weekly expenditure on display advertising. Table 1: Correlation Matrix SA DA SI SC SE DI DC DE SA DA 0.76 SI SC SE DI DC DE Table 1 shows that many variables are highly correlated, especially those related to the same marketing instrument. Display impressions exhibit a 0.98 correlation with clicks and a 0.94 correlation with spend. Therefore, we excluded display clicks and spend from the analysis. 8

12 Table 2 provides summary statistics of our data and Figure 2 illustrates the weekly trend of the resulting series. The decreasing trend present in all variables arises as a consequence of the bank exhausting its advertising budget, and hence decreasing its investments over time to avoid overspending. We incorporate this trend as a non deterministic component of the model, allowing for the other endogenous variables to explain it. Methodology and Analysis We use persistence modeling techniques to capture the complex dynamic interdependencies in online advertising (Dekimpe and Hanssens 1999). Persistence modeling extends multivariate time series methods into the domain of marketing, thereby enabling researchers to model the effects of spillover and feedback dynamics through a system of equations involving marketing actions and consumer response. Persistence modeling is particularly relevant in the context of online advertising as the associated multivariate time series techniques require no stringent a priori restrictions on model structure and allow all variables of interest to affect each other. Persistence modeling involves several steps. A series of tests are used to determine the correct model specification. Granger causality tests are used to identify which variables enter the system endogenously. Unit root tests are done to determine which of the endogenous variables exhibit non stationary behavior and should enter the model in differences. Next, cointegration tests are used to identify stationary linear combinations of non stationary endogenous variables that must be considered in the specification to correct for temporary deviations away from the implied long run equilibria. Granger causality tests, conducted pair wise for variable lag lengths ranging from 1 to 20, suggest that all variables should enter the system endogenously. Figure 3 presents a schematic of the Granger causality results. For example, an arrow from to indicates that is found to Granger cause for at least one of the lag lengths considered. Interestingly, no arrow exists from display impressions to search applications, implying that if display does affect search, the effect travels through search impressions and search clicks. The complex nature of 10

14 The Johansen cointegration trace test identifies three cointegrating relations. These relations can be interpreted as long run equilibrium conditions which may arise as a result of firm budgeting rules or consumer decision processes. Based on the outcomes of the Granger causality, unit root and cointegration tests, we specify a vector error correction model (VEC) with all variables as endogenous. The interpretation of VEC models is particularly interesting from a substantive perspective. Both managers and researchers may expect a long term equilibrium linking search and display applications to a combination of firm control variables (e.g. search and display ad impressions) and consumer actions (e.g. clicks on these ads). Economist Walter Enders states that economic theory abounds with equilibrium theories which, if they involve non stationary variables, require the existence of a combination of the variables that is stationary (2010, p.356). Within marketing, researchers often voice opinions about the necessary intricate relation between firm activity, consumer activity, and purchase action. Error correction models have been used to study the long run impact of a product harm crisis (Van Heerde et al 2007), market share cannibalization by new innovations (Van Heerde et al 2010), and long run sales sensitivity to price changes (Fok et al 2006). The general form of the VEC model with lags is given by equation 1, Δ Δ,, (1) ~0,, In equation (1), is the vector of endogenous variables at time, and are vectors of deterministic components (e.g. intercept, trend), is a matrix of cointegrating relations,,,,, and are parameter matrices to be estimated, and is the covariance matrix of the multivariate normally distributed error terms. The coefficients in,, capture the effects of past changes in the endogenous variables on their current deviations. The coefficients in reflect the speed of adjustment of the endogenous variables towards the equilibrium cointegrating relations defined in. We refine the model further by 12

15 allowing for an intercept in both and. The intercept in the model specification allows for the possibility of a deterministic time trend to exist concurrently with the stochastic one implied by the error correction model. The intercept term in the cointegrating vector is included to account for the initial values of the endogenous variables. The Bayesian Information Criterion identifies a lag length of 1 as optimal. The resulting model specification is indicated in equation (2): Δ Δ Δ Δ Δ Δ Δ Δ Δ Δ , 0 1 0,, 0 0 1,, The parameters are recovered in two steps. First, Johansen s procedure is used to estimate the cointegrating vectors. Then, the first differences of the endogenous variables are regressed on an intercept, their lags and the cointegrating vectors to recover the remainder of the coefficients. Not all the coefficients in this model are identified. In particular standard errors cannot be recovered for, and. Furthermore, an arbitrary normalization is required to identify the remaining coefficients of the matrix. Table 4 summarizes the full set of parameter estimates and asymptotic standard errors. The model exhibits good fit for a model in differences, with individual equation statistics ranging from 0.27 to Portmaneau tests fail to find significant evidence of residual autocorrelation and normality tests fail to reject normality of the residuals. Furthermore, generalized fluctuation tests for structural change fail to find significant evidence of parameter instability. (2) 13

17 The Effects of Search and Display Ads As recommended for multivariate time series models (Sims 1980), we use impulse response functions to analyze the impact of search and display advertising, and assess significance by applying a one standard error band to the impulse response coefficients 4 (Sims and Zha 1999, Dekimpe and Hanssens 1999). Pesaran and Shin (1998) provide a derivation of the generalized impulse response function, which captures the impact of an unexpected shock to the endogenous variables in a VEC model by constructing two forecasts and taking their difference. One forecast takes the shock into consideration, while the other does not. The difference of the two forecasts provides the incremental impact of the shock. Impulse response functions trace the impact of a shock to one endogenous variable through other endogenous variables, thereby providing a cumulative view of all dynamic interactions that take place. For search clicks and display impressions, an unexpected shock represents an investment injection by the firm. In the case of search and display applications, and search impressions, it would imply a scenario in which we would observe an unexpected increase in applications or search viewership, holding display exposure and search clicks unchanged. It is common in practice to make budgeting decisions based on search clicks and CPC, and display impressions and cost per thousand (CPM) impressions. Therefore, we use search clicks and display impressions as the marketing variables of interest and interpret the forecasts that result from their shocks as the effects of increases in marketing investment. We apply one standard deviation shocks to the marketing variables and study their sustenance, implications for performance, and interaction between search and display. Figure 4 presents the sustenance levels of search clicks and display impressions. Sustenance measures the response of a variable to a one standard deviation shock to itself. The 4 We calculate confidence bands for the impulse response functions by simulating 1000 random draws from a multivariate normal distribution with mean zero and covariance matrix equal to the residual covariance matrix of the model, using these draws to perturb the data, and estimating the impulse response functions 1000 times on the resulting simulated datasets. Quantiles of the distributions of coefficients provide an indication of the accuracy of the impulse response functions. We take the 16 th and 84 th percentiles of the empirical distribution to approximate a one standard error band. 15

18 plots suggest that persistent investment and complex consumer transitions between different channels of the conversion funnel lead to sustained levels of long run exposure to marketing. Panel 4a shows that a shock of 4,000 search clicks wears in after 8 10 weeks and stabilizes at about 900 clicks per week in the long run. Display impressions follow a similar pattern according to panel 4b. A shock of 5 million impressions wears in over a period of 7 8 weeks and stabilizes at a sustained level of 1.4 million impressions per week. Figure 4: Sustenance levels of marketing variables The plots in Figure 5 show the performance impact of marketing. The top row captures the impact of initial shocks and persistence in marketing exposure on search applications. Panel 5a shows the impact of search clicks on search applications. A shock of 4,000 clicks generates 15 search applications initially. After a wear in period of 4 weeks, 900 clicks (Figure 4a) generate 26 applications per week (Figure 5a). A smaller number of search clicks is required to maintain a higher level of search applications in the long run, suggesting that the effectiveness of an injection to search advertising increases as it persists over time. 16

19 Figure 5: Performance impact of marketing variables Panel 5b shows the impact of display impressions on search applications. As expected, an initial increase in display impressions does not generate any search applications. However, after a period of two weeks, display impressions positively impact search applications. A sustained level of 1.4 million impressions (Figure 4b) generates about 20 search applications per week (Figure 5b). Consistent with our conceptual framework, display exposure appears to drive consumers to paid search over time. The bottom row of Figure 5 captures the impact of online ads on display applications. A shock to search clicks does not affect display applications (Figure 5c), except for the initial period, which may point to consumers who would have applied through display substituting into the search channel. Panel 5d shows that the effect of display impressions on display 17

20 applications is powerful and immediate. A shock of 5 million impressions (Figure 4b) generates 34 applications immediately. After one week, display applications dip and then stabilize at 28 applications per 1.4 million impressions (Figure 4b) in the long run. To further understand the impact of advertising, we consider the interaction between search and display ads to see how increased levels of display ads may drive search impressions and clicks. Figure 6 plots the impact of a shock in display impressions on search impressions and search clicks. In the short run, we observe a decrease in both search impressions and clicks, which may be driven by consumer substitution across channels. In the long run, a sustained increase in display impressions drives a significant increase in search impressions and clicks, suggesting that display exposure not only increases conversion through search, but also drives search visitation and search clicks. This finding, together with the lack of direct Grangercausality between display impressions and search applications, suggests that display advertising drives search applications through search impressions and clicks. Hence, in calculating the overall impact of display advertising, we must take into account the potential associated increases in costs from search advertising. Figure 6: Impact of display advertising on search funnel progression 18

SELECTION OF BEST KEYWORDS: A POISSON REGRESSION MODEL Ji Li, Rui Pan, and Hansheng Wang ABSTRACT: With the rapid development of the Internet and information technology, consumers have increasingly begun

Is the Forward Exchange Rate a Useful Indicator of the Future Exchange Rate? Emily Polito, Trinity College In the past two decades, there have been many empirical studies both in support of and opposing

Get the latest knowledge from leading global experts. Financial Science Economics Economics Short Courses Presented by the Department of Economics, University of Pretoria WITH 2015 DATES www.ce.up.ac.za

Department of Economics Working Paper Do Stock Market Risk Premium Respond to Consumer Confidence? By Abdur Chowdhury Working Paper 2011 06 College of Business Administration Do Stock Market Risk Premium

16 : Demand Forecasting 1 Session Outline Demand Forecasting Subjective methods can be used only when past data is not available. When past data is available, it is advisable that firms should use statistical

ERC Working Paper in Economic 03/10 November 2003 Testing The Quantity Theory of Money in Greece: A Note Erdal Özmen Department of Economics Middle East Technical University Ankara 06531, Turkey ozmen@metu.edu.tr

Applied Time Series Analysis ANALYSIS OF EUROPEAN, AMERICAN AND JAPANESE GOVERNMENT BOND YIELDS Stationarity, cointegration, Granger causality Aleksandra Falkowska and Piotr Lewicki TABLE OF CONTENTS 1.

Co-movements of NAFTA trade, FDI and stock markets Paweł Folfas, Ph. D. Warsaw School of Economics Abstract The paper scrutinizes the causal relationship between performance of American, Canadian and Mexican

Cointegration The VAR models discussed so fare are appropriate for modeling I(0) data, like asset returns or growth rates of macroeconomic time series. Economic theory, however, often implies equilibrium

Age to Age Factor Selection under Changing Development Chris G. Gross, ACAS, MAAA Introduction A common question faced by many actuaries when selecting loss development factors is whether to base the selected

THE U.S. CURRENT ACCOUNT: THE IMPACT OF HOUSEHOLD WEALTH Grant Keener, Sam Houston State University M.H. Tuttle, Sam Houston State University 21 ABSTRACT Household wealth is shown to have a substantial

What Drives Managerial Use of Marketing vs. Financial Metrics and Does it Impact Performance? An Industry Study hosted by The Paul Merage School of Business University of California, Irvine in cooperation

RE MYTH-BUSTING SOCIAL MEDIA ADVERTISING Do ads on social Web sites work? Multi-touch attribution makes it possible to separate the facts from fiction. Data brought to you by: In recent years, social media

Online Keyword Based Advertising: Impact of Ad Impressions on Own Channel and Cross Channel Click Through Rates Ram Gopal, Xinxin Li, Ramesh Sankaranarayanan University of Connecticut, School of Business

Limitations of conventional marketing mix modelling 1 P.M Cain 1) Introduction The primary function of all marketing mix models is to quantify the sales contribution of marketing activities, with a view

Digital Attribution A practical approach to enhanced digital media performance Agenda Overview: What you first need to know about the data Opportunity and challenges of digital measurement Digital media

Driving Online Traffic and Measuring Offline Phone Conversions Executive Summary We are in the midst of an ever-evolving digital world and every aspect of your online efforts must complement and support

How to Drive Online Marketing through Web Analytics! Tips for leveraging Web Analytics to achieve the best ROI! an ebook by - Delhi School Of Internet marketing 84 84 Table of Content Introduction Chapter1:

1 CHAPTER 11. AN OVEVIEW OF THE BANK OF ENGLAND QUARTERLY MODEL OF THE (BEQM) This model is the main tool in the suite of models employed by the staff and the Monetary Policy Committee (MPC) in the construction

Digital Media and Analytics How to leverage a powerful new set of techniques for digital media buying and program measurement to drive business results CRM Drives Competitive Advantage We believe market

Chapter Vector autoregressions We begin by taking a look at the data of macroeconomics. A way to summarize the dynamics of macroeconomic data is to make use of vector autoregressions. VAR models have become

Analysis of Bayesian Dynamic Linear Models Emily M. Casleton December 17, 2010 1 Introduction The main purpose of this project is to explore the Bayesian analysis of Dynamic Linear Models (DLMs). The main

An Empirical Analysis of Sponsored Search Performance in Search Engine Advertising Anindya Ghose Sha Yang Stern School of Business New York University Outline Background Research Question and Summary of

Online Advertising Defining Relevant Antitrust Markets School of Law and Department of Economics University of California, Berkeley George Mason University, Conference on Search Engines and Online Advertising,

PITFALLS IN TIME SERIES ANALYSIS Cliff Hurvich Stern School, NYU The t -Test If x 1,..., x n are independent and identically distributed with mean 0, and n is not too small, then t = x 0 s n has a standard

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution

7. IS THERE A LONG-RUN RELATIONSHIP BETWEEN TAXATION AND GROWTH: THE CASE OF TURKEY Salih Turan KATIRCIOGLU Abstract This paper empirically investigates long-run equilibrium relationship between economic

Harvard Business School 584-149 Rev. September 29, 1986 Basic Quantitative Analysis for Marketing Simple calculations often help in making quality marketing decisions. To do good numbers work, one needs

DEMB Working Paper Series N. 53 What Drives US Inflation and Unemployment in the Long Run? Antonio Ribba* May 2015 *University of Modena and Reggio Emilia RECent (Center for Economic Research) Address:

University of Wollongong Research Online Applied Statistics Education and Research Collaboration (ASEARC) - Conference Papers Faculty of Engineering and Information Sciences 2011 Is the Basis of the Stock

CPC/CPA Hybrid Bidding in a Second Price Auction Benjamin Edelman Hoan Soo Lee Working Paper 09-074 Copyright 2008 by Benjamin Edelman and Hoan Soo Lee Working papers are in draft form. This working paper