An Uncertain Prognosis for Medicare Auctions: Part II

June 23, 2012

Figure 1. Each of the above curves (except the diagonal line) represents an equilibrium bidding function in a model of the Medicare auctions with 16 bidders, whose costs c are uniformly distributed between $100 and $1000 and whose bids beta are capped at $1000. Any bidding function that lies entirely in the shaded area is also an equilibrium.

The first part of this article (May 2012) presented an auction design about to be implemented nationwide by the Centers for Medicare and Medicaid Services in awarding contracts for durable medical goods. A team of economists and mathematicians---Peter Cramton, Sean Ellermeyer, and Brett Katzman---pointed to flaws in the design. After a review of mathematical auction theory, the article offered a success story for comparison: the Federal Communications Commission's auctions of electromagnetic spectrum licenses in the 1990s.

Erica Klarreich

The thinking behind the Centers for Medicare and Medicaid Services auction design is detailed in the Federal Register of April 10, 2007. The text makes no reference to results in the auction theory literature, and is peppered with sentences that start "We believe that," without offering scientific justification for those beliefs.

As described in the Federal Register, CMS considered four different auction structures. All choose the winners in the same way, starting with the lowest bid and working their way up the bids until there are enough providers to meet the projected demand. (To ensure a sufficient supply of the item in question, a CMS auction typically entails multiple winners.) The auctions differ in how they set the reimbursement price. In addition to the median of the winning bids (the method CMS ultimately chose), CMS considered using the minimum bid, the maximum winning bid, or a more complex adjusted average of the bids, a method it had tested in pilot auctions in 1999.

CMS discarded the adjusted average on the grounds that it made the reimbursement prices higher than the bids and rejected the minimum bid as not "reflective of the actual bids." CMS also rejected the maximum winning bid on the grounds that it would lead to payments that were higher than necessary, because some suppliers were willing to supply the item at a lower cost. The median, CMS wrote, is "representative of all the acceptable bids."

As explained in Part I, CMS made the bids non-binding: A winning bidder who is not happy with the reimbursement price does not have to accept a contract.

In its analysis of the four pricing rules, Brett Katzman observes, CMS incorrectly applied the principle of ceteris paribus, or "all other things being equal," when it assumed that the bids would stay the same no matter which auction design it used.

"The science of auctions---if not simple common sense---tells us that changing the pricing rule will change bidder strategies, and thus the comparisons CMS makes are useless," Katzman says. "Their reasoning was so bad that if my Intro Economics students used it, they would fail my class."

No Clear StrategyIn a median-price auction, it can be advantageous for a supplier to try to game the system by making a bid that's very far removed from his true costs. For example, a bidder with high costs might place an absurdly low bid, just to guarantee that he is included among the winners. His bid probably won't change the median price that much, and if the final price is too low to meet his costs, he can always walk away without signing the contract.

"A low-ball bid is a free option," comments Charles Plott, an economist at the California Institute of Technology.

To understand the ramifications of the median-price auction, Cramton, Ellermeyer, and Katzman set out to compute its Bayesian Nash equilibria, as described in Part I. In the real world, bidders will not necessarily use complex mathematics to identify their Bayesian Nash equilibrium strategies. However, Katzman says, an equilibrium is a proxy for where bidding should evolve to over time, as naive bidders gradually gain experience with an auction format and intuition about what strategies serve them well.

"If an auction has one good, solid, sensible equilibrium, that's a good sign that the auction will work well," Katzman says. "Otherwise, it's hard to tell how people will strategize, because even over time, they probably won't be able to figure out a profit-maximizing strategy."

In doing an equilibrium analysis of the CMS auctions, Cramton, Ellermeyer, and Katzman found themselves in uncharted waters. "No one had ever studied median-price auctions before, because, quite bluntly, no one was ever crazy enough to think they would work," Katzman says.

In this setting, finding a Bayesian Nash equilibrium amounts to looking for a bidding function---one that assigns a particular bid to each possible cost---such that if each bidder is using this function to determine his bid, then no bidder can increase his expected profit by switching to a different bid. Finding such a function boils down to solving a partial differential equation.

Using numerical methods, the research trio found that in the CMS auction design, infinitely many equilibria exist. "It's a really damning property of the Medicare auctions," Cramton says.

Figure 1 shows the various families of equilibria that emerge from the team's model when 16 suppliers, whose costs are uniformly distributed between $100 and $1000, bid for seven contracts. In addition to the curves shown, any "low-ball" function whose graph lies entirely in the shaded area below $100 is an equilibrium. In such a case, the median price is below all bidders' costs, and all will decline the contract; no individual can better his lot by unilaterally bidding over $100, which will simply result in his losing the auction. Even the solid curve dividing the two dotted-line families of equilibria---in some sense the "nicest" of the equilibria---is not a satisfactory solution; the median is a price that some winners are likely to decline, which would lead to a shortage.

In any case, there's no reason to expect that bidders will gravitate toward that equilibrium, given the ever-present temptation to low-ball. In bidding experiments, Plott and colleagues at Caltech have shown that this temptation is all too real.

Poor PerformanceTo test the performance of the Medicare auction design experimentally, Plott and his colleagues asked groups of 12 or 16 volunteers to participate in auctions, with the incentive of modest cash awards for successful bids. Each participant was randomly assigned a cost for producing the (theoretical) item in question; after a practice period, the researchers held CMS-type auctions, in which each participant tried to maximize his profit.

As the theory predicts, bids were all over the map, with many high-cost bidders submitting low-ball bids. "You can see in the bidding that some people are trying to bid high enough to bring up the median without losing the auction, but without systematic coordination, they can't make that happen," Plott says. "When we repeated the auctions, the bids gradually converged to a very low price."

A very low price might sound great in view of ever-increasing medical costs, but the goal of an auction is to find not the lowest possible price, but rather the lowest sustainable price---the one that keeps providers in business and procures sufficient quantities of the items.

By these measures, the auctions conducted by Plott's team failed miserably. Only one in twenty succeeded in procuring the desired number of units, and one in five procured no units at all, as bidders low-balled and then walked away. And many low-cost bidders were shut out by higher-cost bidders who submitted low-ball bids.

"The results say fairly clearly that this auction architecture results in very poor performance in almost any dimension you could imagine," Plott says.

For comparison, Plott's team also tested a "clearing-price" auction, a well-studied design similar to that of the highly successful FCC spectrum auctions. In a clearing-price auction, in which bids are binding, the price is set at the bid of the lowest bidder who loses the auction. Bidding one's true cost is the "dominant" strategy---the best strategy no matter what the other bidders do.

The researchers found that for the most part, the volunteer bidders did figure out and implement this optimal strategy. Every one of the trial auctions succeeded in securing enough supply to meet the buyer's demand (automatically, as the bids were binding), and the low-cost bidders were usually the winners. In most cases, the final auction price was just slightly above the market clearing price---the price at which supply equals demand.

The simple expedient of making bids binding in the CMS auction would solve the problem of supply not meeting demand, and would also pretty much eliminate low-ball bidding---as, indeed, Plott's team found in experiments---as such bidders would run the risk of being held to their absurdly low bids. Yet even as this "fix" closed certain cans of worms, it would open others: In this case bidders would have to wrestle with the risks inherent in bidding at all in an auction in which they could ultimately be forced to accept a price lower than their bids.

In the case of binding median-price auctions, Cramton, Ellermeyer, and Katzman found that no equilibrium function exists. (More precisely, there is an infinite family of functions that satisfy the mathematical requirements to be equilibria, but the functions are non-monotonic, which makes no sense from an economics point of view.)

"In this setting, it's incredibly hard to decide how to bid," Katzman says.

Out of BusinessSo how have the actual CMS auctions performed? The pilot auctions in 1999 (which used the adjusted average method instead of the median bid to set the price) were marked by such wide price swings that CMS set upper and lower limits on allowable bids. "To put an upper limit on bids is not unusual, but to have to put a lower limit on bids when they want a low price should have been an indication to them that something is drastically wrong," Katzman says.

As might be expected, the 2009 auctions resulted in prices substantially lower than those in the earlier CMS fee schedules, and CMS touted the auctions as a notable success. It's hard for outsiders to assess how the auctions performed, however, because two years later, CMS has not revealed the bids. "There's a complete lack of transparency," says Cramton, who is using the Freedom of Information Act to try to elicit information about the auctions.

What he has learned so far suggests that the market structure was "turned upside-down" by the auctions. "The existing providers were largely thrown out and replaced by people who haven't provided before," he says.

If CMS continues to expand its auctions, Cramton predicts that the companies that will survive are the ones willing to accept the low prices and compromise on quality. "I think there's going to be a real race to the bottom."

Cramton is working with members of Congress toward a requirement that CMS make its auctions similar to the spectrum auctions. Meanwhile, he says, there's a moral to the story: "No one would build a bridge without consulting a bridge expert. If you want to design an auction, you have to consult an auction expert. Any expert could have told CMS there were problems with what they were trying to do."