Tag Archives: Justin Wolfers

Post navigation

How do we know the Intrade price was not accurate? Well, the raid wasn’t just executed on a whim. It had been planned for quite some time. Therefore, the true likelihood must have been much higher than the four or five percent chance the market was telling us.

Many people knew of the plans, albeit they were very high ranking, sworn-to-secrecy types. Since the market did not reflect the potential success of the planned raid, either the market was inefficient or the market, in an aggregate sense, did not possess enough information to make a reasonably informed prediction. In this case, I have to believe the market participants (every last one of them) knew next to nothing about the outcome being predicted.

The Intrade market prediction was nothing more than an aggregation of guesses. This is very different from an accurate prediction (based on calibration) that turns out to be wrong.

Markets such as these have no use, whatsoever, in decision-making. The useful information was that gathered by the SEALs and other secret services, and that was the information provided to the real decision-maker, The President. I would argue that these types of markets have no place as betting markets either. There is no way to test the calibration, so we don’t know whether they are “fair” markets (unlike the accurate calibration of horse races and casino games).

In other words, stop wasting our time operating and analyzing these markets. They are never going to be useful.

How do we know, now, that Intrade&#8217-s market price was not an accurate estimate of the probability bin Laden was killed or captured by September 2011? Is an prior estimate of 50 percent likelihood that a tossed coin will come up heads wrong if the coin comes up as &#8220-100 percent&#8221- heads (and not half-heads and half-tails)?

I&#8217-m not buying Chris&#8217-s implied definition of success and failure.

However, one might ask Robin Hanson about what the Intrade market&#8217-s performance implies about the usefulness of his Policy Analysis Market idea.

Note that I was contrasting the InTrade-Bin-Laden failure with the high expectations set by Robin Hanson, Justin Wolfers and James Surowiecki.

Also, other than statisticians, most people don&#8217-t have a probabilistic approach of InTrade&#8217-s predictions. That&#8217-s the big misunderstanding, which is one part of the big fail of the prediction markets.

In summary, &#8220-momentum&#8221- can exist, but the places where you&#8217-ll see it is in races where current public opinion is out of step with best predictions. The mere information that a race has a 5-point swing is not enough to predict a future shift in that direction. As Nate emphasizes, such a prediction is only appropriate in the context of real-world information, hypotheses of &#8220-factors above and beyond the direction in which the polls have moved in the past.&#8221-

The reason prediction markets are not widely used in business is that their many boosters (Robin Hanson, James Surowiecki, Justin Wolfers, etc.) have exaggerated their usefulness. Just because they are objective in their wisdom does not mean that they are very useful.

Objectivity is over-rated. This is a painful lesson for the handful of young startups who swallowed the prediction market myth. Next step: the dead pool.

You are giving yourself WAY too much credit. Siskel and Ebert successfully predict these awards 100% year after year. This isn’t a difficult thing to predict. Predicting something like the NCAA tourney, that would be an accomplishment, but if you look at rankings and your prediction market, you will fail just as much as the average bracket.— Jason

Justin Wolfers has escaped the Overcoming Bias purge, it seems. Justin Wolfers&#8217-s 4 posts (published in 2007) remain archived on what is now Robin Hanson&#8217-s QUOTEpersonal blog UNQUOTE. By contrast, Eliezer Yudkowsky&#8217-s posts written for Overcoming Bias now redirect to locations at Less Wrong.

Chris, Eliezer was not “purged.” He requested to have his old posts moved to Less Wrong. No one else has made any similar request.

Eliezer was not “expelled”- he choose to move in order to build a community at Less Wrong using fancy comment karma software. The folks who wrote software for his new site also wrote the code at my new site.

Prediction markets can yield valuable insight into the dynamics of political campaigns, a conclusion we&#8217-ve drawn from years of intensive study and research. We&#8217-ve even proselytized about the value of these markets, extolling their ability to yield sharper insights than pundits or polls. […]

-

If this statement were true,

Justin Wolfers&#8217- columns at the WSJ would have been linked to by the blogging political experts. They never were.

The blogging political experts would have adopted the prediction market tool (over than just quoting the InTrade prices out of curiosity). They never did.

-

Both the mystification of the prediction markets (mudding the primary indicators into commentary- suggesting that the traders&#8217- anticipations are always sound) and their demystification (listing the primary indicators) don&#8217-t do the trick: Economic science should be able to tell us whether the prediction markets on 2008 US elections are of high social utility, and whether other kinds of prediction markets are of higher social utility. I am not satisfied by what I have been reading, as of today. The prediction markets are rather a tool of curiosity, as of today, not much a tool of forecasting. The prediction markets are not used as a tool by the experts &#8212-by &#8220-the experts&#8221-, I mean all the experts but the prediction market experts (who are expert in nothing else than pumping up the prediction markets): the political experts, the financial experts, the management experts, the oil production experts, the credit experts, the health care system experts, the automobile market experts, the wine market experts, the web technology business experts, the web advertising experts, the medical drug experts, the foreign affairs experts, the military experts, the aviation industry experts, the condom industry experts, the restaurant industry experts, etc.

-

APPENDIX

Robin Hanson:

[I]nfo value [] is the added accuracy the markets provide relative to other mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining the markets, relative to the cost of other mechanisms. A highly accurate market has little value if other mechanisms can provide similar accuracy at a lower cost, or if few substantial decisions are influenced by accurate forecasts on its topic.

-

PREVIOUSLY: See Robin Hanson&#8217-s take on Google&#8217-s enterprise prediction markets.