Tag Archives: David Pennock

Post navigation

What&#8217-s the difference between predict a lot y Bolsa de predicciones? they have the same things or at least very similar. ?Are you the same? They have predictions, groups, events etc. the only difference is the language- BP is in spanish.

This is my answer.

The main difference is that on bolsadepredicciones (and other prediction markets like crowdcast, inklingmarkets, intrade, newsfutures, hubdub RIP, and WorldCupX.com &#8212- a new site with a nice/fresh look and good social hooks) every prediction is independent (separate) and there are at most a few thousand.

In predictalot, all predictions are interrelated (predicting Spain to win automatically increases Spain&#8217-s odds of reaching the knockout round) and there are millions and millions of predictions possible, far more than other sites.

Predictions are flowing in about every three minutes. Here are some from the last half hour:

Chile will finish second in their group. Current odds: 33.50%.Spain will finish first in their group. Current odds: 52.48%.Spain will play Portugal in the knockout stage. Current odds: 48.77%.Roger joined the group &#8216-inetco&#8217-Spain will advance further than Greece. Current odds: 76.93%.Spain will not advance to the Knockout Stage. Current odds: 23.13%.Spain will advance to the Knockout Stage. Current odds: 77.86%.Both United States and Uruguay will advance to Quaterfinals. Current odds: 60.04%.

Barack Obama might do it, according to VentureBeat on NYTimes.com: &#8220-President Barack Obama will likely be busy this week [but]&#8230- maybe he’ll be able to sneak a peek at Predictalot on his BlackBerry between meetings.&#8221-

Most published papers on prediction markets (there aren&#8217-t many) paint a wildly rosy picture of their accuracy. Perhaps it is because many of these papers are written by researchers having affiliations with prediction market vendors.

Robin Hanson is Chief Scientist at Consensus Point. I like his ideas about combinatorial markets and market scoring rules, but I think he over-sells the accuracy and usefulness of prediction markets. His concept of Futarchy is an extreme example of this. Robin loves to cite HP&#8217-s prediction markets in his presentations. Emile Servan-Schreiber (Newsfutures) is mostly level-headed but still a big fan of prediction markets. Crowdcast&#8217-s Chief Scientist is Leslie Fine- their Board of Advisors includes Justin Wolfers and Andrew McAfee. Leslie seems to have a more practical understanding than most, as evidenced by this response to the types of questions that Crowdcast&#8217-s prediction markets can answer well: &#8220-Questions whose outcomes will be knowable in three months to a year and where there is very dispersed knowledge in your organization tend to do well.&#8221- She gets it that prediction markets aren&#8217-t all things to all people.

An Honest Paper

To some extent, all of the researchers over-sell the accuracy and the range of useful questions that may be answered by prediction markets. So, it is refreshing to find an honest article written about the accuracy of prediction markets. Not too long ago, Sharad Goel, Daniel M. Reeves, Duncan J. Watts, David M. Pennock published Prediction Without Markets. They compared prediction markets with alternative forecasting methods for three types of public prediction markets: Football and baseball games and movie box office receipts.

They found that prediction markets were just slightly more accurate than alternative methods of forecasting. As an added bonus, these researchers considered the issue that prediction market accuracy should be judged by its effect on decision-making. So few researchers have done this! A very small improvement in accuracy is not considered material (significant), if it doesn&#8217-t change the decision that is made with the forecast. It&#8217-s a well-established concept in public auditing, when deciding whether an error is significant and requires correction. I have discussed this concept before.

While they acknowledge that prediction markets may have a distinct advantage over other forecasting methods, in that they can be updated much more quickly and at little additional cost, they rightly suggest that most business applications have little need for instantaneously updated forecasts. Overall, they conclude that &#8220-simple methods of aggregating individual forecasts often work reasonably well relative to more complex combinations (of methods).&#8221-

For Extra Credit

When we compare things, it is usually so that we can select the best option. In the case of prediction markets it is not a safe assumption that the choices are mutually exclusive. Especially in enterprise applications, prediction markets are heavilydependent on the alternative information aggregation methods as a primary source of market information. Of course, there are other sources of information and the markets are expected to minimize bias to generate more accurate predictions.

In the infamous HP prediction markets, the forecasts were eerily close to the company&#8217-s internal forecasts. It wasn&#8217-t difficult to see why. The same people were involved with both predictions! The General Millsprediction markets showed similar correlations, even when only some of the participants were common to both methods. The implication of these cases is that you cannot replace the existing forecasting system with a prediction market and expect the results to be as accurate. The two (or more) methods work together.

Not only do most researchers (Pennock et al, excepted) recommend adoption of prediction markets, based on insignificant improvements in accuracy, they fail to consider the effect (or lack thereof) on decision-making in their cost/benefit analysis. Even if some do the cost/benefit math, they don&#8217-t do it right.

Where a prediction market is dependent on other forecasting methods, the marginal cost is the total cost of running the market. There is no credit for eliminating the cost of alternative forecasting methods. The marginal benefit is that expected by choosing a different course of action than the one that would have been taken based on a less accurate prediction. That is, a slight improvement in prediction accuracy that does not change the course of action has no marginal benefit.

Using this approach, a prediction market that is only &#8220-slightly&#8221- more accurate, than those from alternative forecasting approaches, is just not good enough. So far, there is little, if any, evidence that prediction markets are anything more than &#8220-slightly&#8221- better than existing methods. Still, most of our respected researchers continue to tout prediction markets. Even a technology guru like Andrew McAfee doesn&#8217-t get it , in this little PR piece he wrote, shortly after joining Crowdcast&#8217-s Board of Advisors.

A recruiter who left Google last year says that the company [= Google] had maintained a “do not touch” list of companies including Genentech and Yahoo, whose employees were not to be wooed to the Internet search giant.

That revelation could be significant in light of this week’s disclosure that the U.S. Justice Department is investigating whether Google, Yahoo, Apple, Genentech and other tech companies conspired to keep others from stealing their top talent. […]

This clarification from our good doctor David Pennock shows how indispensable this research scientist is to small people like us. Without David Pennock, the field of prediction markets would collapse like a castle of cards. Ha! ha! ha!&#8230-

As a blogger, WeatherBill presents a difficulty: How to you categorize it? I fill it in these 4 categories:

insurance (obviously)-

finance (because it is hedging)-

exchanges (because of what our good doctor Pennock said above, see the title)-

[…] Prediction markets are gaining interest because the Internet allows greater worldwide access to them, as well as to the ever-increasing amount of data stored on any topic imaginable (which theoretically allows participants to make more informed predictions, individually and in aggregate). These factors, plus the enormous amount of computing power that will make it possible to instantly calculate exponentially small odds, are stimulating new research on advanced computational models in prediction markets. These models could be capable of analyzing entire events such as the annual NCAA collegiate basketball tournament, which begins a 63-game schedule with 263 possible outcomes by the tournament&#8217-s end. […]

Growing opportunities in internal private-sector prediction markets are also revealing divergent philosophies among the markets&#8217- designers. Many of the public markets feature price-adjustment algorithms built around answering discrete multiple-choice outcomes, such as which candidate will win an election or if a product will launch in month x, y, or z. […]

IEM steering committee member Thomas Rietz, a professor of finance at the university, says the aggregate zero-risk design of the IEM allows the markets to perfectly reflect the aggregate forecast opinions of its participants. By aggregate zero-risk, Rietz explains that when a trader enters a particular bilateral (either/or) market, he or she must buy one share of each choice, called a bundle, for a total cost of $1. If the trader holds the bundle until the market concludes, there is neither profit nor gain. If the trader guesses the outcome successfully, and sells the losing unit of the bundle to another trader while the market is running, he or she picks up the original $1 bet plus whatever price was agreed upon for the losing share that was sold. If the trader chooses to hold onto the loser and sell the eventual winner, however, they will incur the $1 loss at payout time. At any given time, the number of eventual winning shares and losing shares is equal and held by the traders. So, the university bears no counterparty risk and there is no need to provide hedging margins that irrationally affect outcomes. &#8220-The price you would be willing to buy or sell for today is your expectation of its value in the future—the prices can be directly interpreted as a forecast,&#8221- Rietz says. &#8220-In ordinary futures markets, there is a long-lasting debate, going back to John Maynard Keynes in the 1930s, over whether prices can legitimately be used as forecasts, and it all hinges on whether or not people demand a return or face a risk in aggregate when they&#8217-re investing in these contracts.&#8221- […]

One enduring research problem on combinatorial markets is mitigating the effects a virtually unlimited spectrum of outcomes will have on creating markets that are so thin in trades they do not serve their purpose of aggregating information. In such markets, which might bear a resemblance to an enterprise prediction market in that there are not enough participants to provide a statistically valid spread of opinion, Pennock says a market-maker algorithm might serve as a price setter within widely acceptable limits. &#8220-I believe that approximation algorithms will be fine for the market maker, because people don&#8217-t really care about making bets on things that are incredibly unlikely, like 10?6 chance,&#8221- Pennock says. &#8220-But as long as you&#8217-re betting on something with a 10% chance of happening, we&#8217-ll be able to approximate pretty quickly with a market-maker price.&#8221- […]