Header Bidding Unleashed A Huge Infrastructure Problem And Ad Tech Will Either Sink Or Swim

As header bidding grows, ad exchanges are sending more ad impressions per second and DSPs must evaluate more impressions than ever. The massive uptick in volume, which far outpaces programmatic revenue growth, is straining ad tech infrastructure.

With more queries to sift through, it costs more to buy the same ads. And as a result, only the biggest and strongest vendors will survive.

In the second half of last year, demand-side platforms (DSPs) started seeing abnormal increases.

DataXu, for example, normally sees 40% annual increases in queries, but it experienced a 100% increase in 2016, which it attributed to header bidding. MediaMath saw a 20% rise in queries last year, with averages fluctuating between 5 million and 6 million queries per second.

"The supply side sees billions to tens of billions of impressions,” said Mike Driscoll, CEO of analytics company Metamarkets. “But when you get to the buy side, it's tens to hundreds of billions a day in programmatic markets, which can see 500 billion unique requests a day.”

“If header bidding comes in and puts a multiplier on that, and requests go up two to three times, you’re talking trillions a day,” Driscoll added.

Both sides are pressured to keep up, said Brian Stempeck, chief client officer at The Trade Desk. It costs DSPs to bid on all the ads, and it costs SSPs to send those bids to the DSP, he explained.

These rising costs will hit mid-size and small DSPs and SSPs the hardest.

Smaller DSPs that can’t handle the query volume will have to dial down indiscriminately how many impressions they listen to, forcing them to fish from a smaller pool, which will hurt performance and send them into a downward spiral.

And as larger DSPs and SSPs optimize their connections to each other to manage the uptick, less competitive buyers and sellers will be pruned away.

Natural Selection

DSPs and SSPs have two options to deal with the rise in impressions due to header bidding: increase server capacity, or find a way to reduce how many ad impressions they must process.

The simple but inefficient way to deal with more queries per second is to spin up more servers.

Some CTOs used capital budgets late last year to deal with the rise in queries, said Neal Richter, a consultant for Hebbian Labs and former CTO of Rubicon Project.

But scaling up hardware use is expensive. “If they need to buy two times the hardware, that is a real cost,” said Igor Vezmar, partner at tech and media consulting firm Konstrukt. “Your margins get squeezed and you can’t maintain it over time.”

And adding servers doesn’t mean adding value. Though DSPs are seeing a flood of new impressions, many of them are selling the same thing. A publisher with three header bidding partners will have three exchanges sell its inventory, tripling the amount of impressions a DSP must evaluate and tripling the listening cost.

“Advertisers have historically judged [DSPs] on their QPS [queries per second] number,” Richter said. “But if it’s QPS with duplication, what value does that have? Advertisers need to ask for deduped QPS.”

Instead, many DSPs are devoting resources to deduplicating impressions to avoid spending millions in server fees.

“You want to filter out what has low value for you as quickly and cheaply as possible,” Richter said.

Discovering the most efficient route to an impression is called pathfinding, smart routing or traffic shaping, depending on which DSP you’re talking to. Each tactic is unique, and each DSP weeds out low-value impressions differently.

“You can get the same publishers across four or five exchanges, so we can pick and choose who we want to do business with,” said DataXu CTO Bill Simmons. “Whoever has the most transparency and the most fair and well-functioning connections will get the majority of DSP traffic.”

DSPs can also identify a path that avoids the worst of the buy-side fees charged by exchanges – the same fees that led The Guardian to sue Rubicon Project.

“If I’m buying the same impression from three different SSPs, I have to find ways to differentiate them,” Stempeck said. “What SSP will be most transparent with me about what cut they are taking? And if one is taking 50% and one is taking 5%, and I have information to support that, I’m going to bias the 5% one.”

MediaMath created a whole product, dubbed Curated Market, to offer just a slice of available inventory, said Tanuj Joshi, VP of strategic media enablement at MediaMath. Billed as a way to find brand-safe inventory, the product looks for low-fee, direct paths to publishers; 70% of the DSP’s clients use it.

But not all DSPs have the resources to smartly filter out low-value impressions.

DSPs who don’t have the budget to deal with the rise in impressions due to header bidding might decrease their QPS indiscriminately, by asking an exchange to send only 10% of what they have. But performance will suffer, as they miss catching high-value users at the right time or the right price.

What Happens To DSPs?

Growing infrastructure costs and competition will kill off the smaller DSPs and SSPs.

Choicestream, for example, laid off most of its staff last month. Other DSPs and SSPs are still quietly struggling or looking to exit or reinvent their business.

Some small DSPs will be able to adapt by partnering with larger players who do the heavy lifting to connect to exchanges at scale.

Many large DSPs have had conversations with smaller DSPs or ad networks who want to use their own tech on the bigger vendor’s infrastructure.

The Goodway Group, which advises agencies on programmatic, decided from the outset that it didn’t want to deal with infrastructure. Instead, it built its own tech, algorithms and analytics on top of The Trade Desk’s API.

Other small DSPs will move to that model, said Goodway Group COO Jay Friedman.

“There is zero reason on the planet to launch a DSP today,” he said. “There are plenty of DSPs you can go build on top of and use their existing infrastructure.”

Beeswax CEO Ari Paparo founded his company two years ago on this very insight, and said he’s seeing inquiries from some of these smaller DSPs.

“You can build a DSP on us in the hour,” he said. “The economics of running a small DSP were never good, and are getting worse. It doesn’t make sense to scale up infrastructure, unless you are going to be quite large in the long term.”

Smaller buying platforms also seek to borrow algorithms from larger vendors. Criteo told AdExchanger that some DSPs have inquired about using its proprietary deduplication technology.

But the situation will get worse for DSPs that don’t partner, because the SSPs might start cutting them off.

“If you are an SSP, do you want to send out that impression to DSP No. 51 that spends only $10,000 a month?” Stempeck said. “The SSPs are sending all their bids to the bigger DSPs, and the smaller entrants don’t see as many impressions.”

And the larger players, whose infrastructure and capital budgets can handle the load, will see more of the market move to them, Richter said.

He doesn’t see how the 250 to 300 long- and mid-tail DSPs survive, unless they specialize or manage what inventory they look at closely.

What Happens To SSPs?

In the short term, header bidding was great for many SSPs, as they could finally compete against Google AdX and SSPs with top access like Rubicon Project.

SSPs don’t have exclusive access to inventory anymore. Instead, the best price wins.

And that makes it easier for DSPs to route around a player they don’t like without suffering any loss in performance.

DSPs who begrudgingly accepted yield-producing tactics like dynamic floors or first-price auctions from SSPs have less patience for games. They can simply pack up and go elsewhere for the same thing.

SSPs have at least one way to hold on to their position: kickbacks. One DSP reported being offered such a deal, which it declined.

To ensure they aren’t cut off from demand, SSPs will have to become more transparent about fees and lower them.

Goodway Group’s Friedman said current SSP take rates are unsustainable

And second-tier SSPs with mediocre tech, bad marketplace quality and no unique inventory will just get turned off. And publishers will remove partners that no longer provide value.

“You should see second-tier SSPs suffer. Because they are getting less demand, they will have less yield to give to publishers, and then get less demand,” Paparo said. “It’s a spiral to the bottom.”

But as both DSPs and SSPs suffer due to the duplicative wrath unleashed by header bidding, many in the industry see bright spots.

Competition will remove players with fraud, bad tech and high fees. Companies will consolidate as economies of scale kick in. Infrastructure costs create economies of scale, and the barrier to entry in ad tech will be that much higher.

@Ben Brian's article is interesting too, but I think it's unrelated. His is a call to action for DSPs to cut out SSPs who manipulate auction dynamics (most commonly those who run a 1st-price auction). This article is about how publishers bringing on more SSPs through header-bidding will naturally effect SSP/DSPs. Both are good reads imo.

True, I think the correlation I saw was the impact to the ecosystem of header bidding. Both articles talk about what happens to buy side and sell side platforms when the path to supply is now overlapping. But you're right they have different angles; I just saw Brian's post as especially relevant to the 'what happens to DSPs' section, since it's essentially a detailed blueprint on what he thinks they should do.

Thank you for this very interesting article. This strong increase in volume is actually a very good opportunity for SSPs to differentiate from other players by sending to DSPs only clean and deduplicated traffic that they have a good enough probability of filling and winning, at the right price and with enough remaining time to actually reply to the bid requests.

In smaller markets header bidding is more a challenge than a solution. Why connect multiple SSPs if all demand is coming from 20 local buyers. An holistic, agnostic and transparant platform is the best solution for both buyers and sellers.

While header bidding has great promise for maximizing the most value out of every impression, there are some downsides for publishers that often get overlooked.

Just as DSPs see a huge spike in duplicate bid requests from multiple SSPs, users see a dramatic increase in ad calls per page. It's not uncommon to get 50 or 100 Ghostery calls on "premium" sites. This results in really poor user experiences.

Beyond that, the efficiency and yield gains are misleading. Savvy DSPs will exploit this to only bid on the cheapest possible impression. Publishers now have to manage many SSP relationships and setups, straining their yield management teams.