recovered from the data. As a result,
this approach allows one to establish
the bounds for auction welfare bypassing complex computations required in
the approaches previously used in the
economics literature by combining statistical inference and results from the
algorithmic game theory. The Syrgkanis
and Tardos37 approach may potentially
be applied to other bounds, for example,
comparing the welfare of a given auction
mechanism to the welfare of another
auction mechanism (instead of the
welfare of the optimal allocation). We
believe that such an analysis is an important direction for future research.

Advertising markets with learning
agents. Real-world advertising platforms are complex systems with thousands of bidders who compete in the
same auction with bidders dynamically changing their bids, entering and
exiting auctions. In this case, information requirements for bidders to
derive their Bayes-Nash equilibrium
profiles are truly impractical since
they are required to form beliefs over
the actions of all of their thousands
of opponents, as well as the dynamic
adjustment of auction parameters by
the advertising platform.

In practice, most bidders in these
large advertising platforms use algorithmic tools that allow them to automatically and dynamically update their
bids for multiple ads and advertising
campaigns. The algorithmic solutions implemented in these tools take
the advertisers goals (in terms of yield
of auction outcomes) as inputs, and
adjust bids using dynamic feedback
from the auction outcomes. Such
implementations can be associated
with algorithmic learning, where the
bidding strategy is treated as the goal of
online statistical learning procedure.

Recent work by Blum et al., 8, 9 Blumand Y. Mansour, 10, 11 Caragiannis et al., 12Hartline et al., 19 Kleinberg et al., 22Roughgarden, 36 and Syrgkanis andTardos37 shows that some of the worstcase outcome properties of full infor-mation pure Nash equilibria extendto outcomes when all players use no-regret or low-regret learning strategies,assuming the game itself is stable. Theassumption that players use low-regretlearning to adjust their strategies isattractive for a number of reasons.First, low-regret learning outcomewelfare of the optimal allocation (OPT)and the welfare of the auction A that isactually implemented,WELFARE(OPT) £ EPoA(A; D) ×WELFARE(A). ( 5)

Thus, the EPoA is the characteristic of
an auction that allows us to measure
the efficiency of the current auction
mechanism without estimating (the
set of) values of the bidders.

The empirical price of anarchy (and,
subsequently, the bound for optimal
welfare) is defined for the true distribution of auction outcomes D(×).
Then, the idea is to replace the true
distribution of auction outcomes with
empirical distribution (×) of auction
outcomes observed in the data. Given
that the empirical distribution is a
consistent estimator of the true distributionc the bound for the welfare constructed using EPoA (A; ) will bound
the actual auction welfare with probability approaching one (as we get more
samples from the distribution of auction outcomes D(×) ). We call EPoA (A; )
the estimator for EPoA.

We note that even the estimator for
EPoA is defined as a potentially complex
constrained optimization problem. It
turns out that it is possible to avoid solving this problem by invoking the revenue
covering approach. The revenue covering
approach is based on establishing the
ratio of the actual auction revenue and
maximum payment that can be extracted
from participating bidders in equilibrium. This ratio can be used to establish a
simple bound for EPoA. We now describe
this approach in more detail in application to the sponsored search auction.

Consider the sponsored search
auction model with uncertainty as we
described in detail. We can define the
average price per click for bidder i with
bid bi as ppci(bi) = TEi(bi)/Qi(bi). The typical function that provides the expected
number of clicks as a function of the
bid Qi(bi) in the sponsored search auction is continuous and monotone. As
a result, we can construct its inverse
that specifies the bid that is
needed to get the expected number of
clicks z. Then, the average price per click

c Formally, the infinity normconverges to zero in probability under mildassumptions regarding D(×).can be redefined in terms of expectedclicks as , whichis the average price per click that bid-der i getting z clicks pays. Functionis called the thresholdbecause it corresponds to the mini-mum price per click that bidder ineeds to pay to get z clicks in expec-tation. The cumulative threshold foragent i who gets Qi expected clicks isTi(Qi) can be inter-preted as the total payment that bidderi would have paid, had she purchasedeach additional click at the averageprice of previously purchased clickswhen she purchases a total of Qi clicks.

Definition 1. Strategy profile s of auction A (defining the mapping from bidders’ values into their bids) is m-revenue
covered if for any feasible allocation

mRevenue (A, s ) ³ Si Ti(Qi). ( 6)

Auction A is m-revenue covered if for
any strategy profile s, it is m-revenue
covered.

The inequality Equation ( 6) can be
defined in expectation over the realizations of possible equilibrium profiles
and all thresholds corresponding to the
realizations of bid profiles. Then, the
expected revenue from the auction can
be measured directly by the auction platform (as a sum of payments of bidders per
auction over the observed auction realizations). The thresholds can be computed
via simulation from the observed auction
realizations given that the allocation and
pricing rule is controlled by the auction
platform, and thus empirical equivalents
of TEi(×) and Qi(×) can be computed for
each bid bi. As a result, the platform can
compute the revenue covering parameter
for given auction mechanism A.

The next result, developed in
Syrgkanis and Tardos, 37 takes revenue
covering parameter and provides the
EPoA for the auction mechanism A.

Theorem 1. The welfare in any m-revenue
covered strategy profile s of auction A
is at least a -approximation to the
optimal welfare. In other words, EPoA
(A; D) .

The estimator for the EPoA is then
obtained by replacing the true revenue
covering parameter m with its estimator