Finance Celebrated, Ignored

There is no way to predict the price of stocks and bonds over the next few days or weeks. But it is quite possible to foresee the broad course of these prices over longer periods, such as the next three to five years. These findings, which might seem both surprising and contradictory, were made and analyzed by this year’s [Economics Nobel] Laureates, Eugene Fama, Lars Peter Hansen and Robert Shiller. (more)

Today news outlets of all sorts will report on this year’s Nobel prize in economics, which goes to three giants of finance. There will be lots of commentary on subtle results in finance and what they mean for stock returns and financial crises. But one thing you might not notice in all that commentary is a simple fact that seems too obvious and widely accepted among finance experts and the Nobel committee to be worth mentioning: speculative markets generally do an excellent job of aggregating information.

Among the many theories still debated to explain the wide variety of odd patterns in market prices, virtually none of them have much to do with speculators failing to act on cheaply available information. Why? Because so many folks in finance have long tried and failed to find satisfactory explanations along those lines, and have basically given up. This isn’t to say that we never see price errors after the fact, along with previously available info not very well considered. (See this nice example.) The point is that there are few interesting or consistent pattern of such things; mostly speculators just make excusable mistakes about what info to consider.

Yet even though this simple fact seems too obvious for finance experts to mention, the vast majority of the rest of news coverage and commentary on all other subjects today, and pretty much every day, will act as if they disagreed. Folks will discuss and debate and disagree on other subjects, and talk as if the best way for most of us to form accurate opinions on such subjects is to listen to arguments and commentary offered by various pundits and experts and then decide who and what we each believe. Yes this is the way our ancestors did it, and yes this is how we deal with disagreements in our personal lives, and yes this was usually the best method.

But by now we should know very well that we would get more accurate estimates more cheaply on most widely discussed issues of fact by creating (and legalizing), and if need be subsidizing, speculative betting markets on such topics. This isn’t just vague speculation, this is based on very well established results in finance, results too obvious to seem worth mentioning when experts discuss finance. Yet somehow the world of media continues to act is if it doesn’t know. Or perhaps it doesn’t care; punditry just isn’t about accuracy.

Even these Nobel prize winners, now that they have the attention of the world for a few days, won’t bother to mention how we could use finance to effectively answer most of the non-finance questions we commonly debate. These academic finance experts won’t even think to discuss how the academic finance community itself could use speculative markets to create and disseminate accurate estimates on important disputes in academic finance. Is that because they don’t notice, or don’t care, or what?

1) Do you agree with the quoted assertion that it is possible to foresee the course of stock prices over a timeframe of 3-5 years? If so, is this an example of speculative markets *not* doing an excellent job of aggregating information?
2) Do you think that speculative betting markets would still do an excellent job of aggregating info if their participants did *not* do all of that thinking/discussing/debating?

http://overcomingbias.com RobinHanson

1) Yes, but if you’ll read the coverage you’ll see that the prize is being given for other explanations of that, not info aggregation failure explanations.
2) Of course people need to think and discuss. The question is what forum to use, and what summaries others should use to see what they’ve learned.

arch1

Re: 2), thanks for correcting my misinterpretation. You might consider tweaking your presentation to more explicitly make this point (i.e. that you’re advocating betting markets not as an alternative to thought and discussion, but as a context in which they can be more effective and via which their upshot can be better distilled).

IMASBA

“speculative markets generally do an excellent job of aggregating information.

But by now we should know very well that we would get more accurate estimates more cheaply on most widely discussed issues of fact by creating (and legalizing), and if need be subsidizing, speculative betting markets on such topics. This isn’t just vague speculation, this is based on very well established results in finance”

The market does not predict prices, it sets prices. The price mechanism itself does not predict anything.

Sure you could let people speculate based on stats of past results but that’s basically just guessing (it’s not even the market at work) for money and the stats of past results will be of poor quality and low quantity because most things exciting enough to want to predict are relatively rare and complicated events.

Btw: accurate prediction runs into infinite regress problems (speculators know the prediction which changes their actions which changes the prediction which the actions of speculators which changes the prediction, and so on…), unless speculation is pointless to begin with (changing the actions of the speculators doesn’t change the outcome) and that calls into question the whole point of speculative prediction markets.

efalken

Lots of useful signals are dismissed because they can’t be managed by a central authority and often give unwelcome answers. Then it’s just a matter of straw men arguments–exceptions, counterexamples–to ‘prove’ this is a principled, scientific, objective view. Think about lie detector tests: they are considered useless by the courts because they have flaws, but surely no less so than witness testimony.

Shiller and Fama have aggressively pushed various market indices piecemeal (eg, Ken French’s website is an invaluable resource based on Fama’s work, there is the Case-Shiller housing index).

RobS

Hi Robin. May I ask a basic question? What is the scope of questions that you feel speculative markets could help with? You say they could “effectively answer most of the non-finance questions we commonly debate”. What questions would this include and exclude? I take it that such markets wouldn’t help with open ended or hypothetical questions, or with questions that involve subjective opinions. My assumption is that you mean questions that can be verified to the satisfaction of impartial observers in a finite timespan — essentially debates about whether stated events do or don’t happen. To me this seems quite limited — it includes sports leagues results and weather predictions but not much else outside traditional markets. Am I being limited in my thinking? If so, can you give me some other examples of useful answers that markets could provide but currently don’t or aren’t allowed to? Thanks.

http://www.gwern.net/ gwern

Google ‘futarchy’.

RobS

Thanks for the link. It says that in a futarchy, “elected officials define measures of national welfare and prediction markets are used to determine which policies will have the most positive effect.”

I’m still confused about how this works in anything but the most simple cases (e.g. Will this person get elected? Will GDP grow or fall this year?). Wouldn’t most interesting questions be hypothetical ones as opposed to simple predictions, meaning that we won’t be able to satisfactorily prove who is right even when the evidence is in? (Will raising taxes increase GDP? Is staying firm on the debt ceiling a vote winner?). If these are the questions we care about I am confused as to how prediction markets help.

VV

speculative markets generally do an excellent job of aggregating information.

Actually, no. The fact that long-term market prices are predictable implies that the market is inefficient: in principle an investor could exploit such predictions to make a gain. If all investors did the same, the market prices would be chaotic.

http://www.gwern.net/ gwern

> because they don’t notice, or don’t care, or what?

Perhaps they don’t want to draw the lightning. I believe… Cowen? in his blog post today on Fama remarked that he was pleased the Nobel committee had the guts to give the award to Fama since efficient-markets is, like IQ, extremely unpopular outside the relevant experts’ profession. And I believe you have some fairly recent personal experience with reactions to markets on nontraditional topics, do you not?

http://overcomingbias.com RobinHanson

Nobel prize winners are famous for using their new pulpit to pontificate on lots of subjects, often well outside their areas of expertise. They are the least afraid of lighting than at any other point in their career, or in the career of any academic. This is their big chance to say something important and be heard. So what they choose to talk about says a lot about their priorities. Yes, speaking up on some subjects could detract from the prestige they could bring to bear on other subjects. I claim this topic is very important, and so lament that they don’t seem to think so.

IMASBA

Well, perhaps they don’t know about futarchy or do not think it’s desirable. Smart people disagree on things, it happens, I kid you not, there are even smart people who write blogs about their dreams of a world that has almost all personhood rights and an above-subsistence-level living standard for an increase in total productivity.

http://www.gwern.net/ gwern

Watson, a Nobel prize winner among prize winners – spoke up about an important issue. How well was he heard and does his experience justify fear on the part of others? What makes prediction markets, given the highly predictable and fierce backlash, the highest marginal return for Nobel prize winners?

http://entitledtoanopinion.wordpress.com TGGP

Watson made an off-the-cuff response in an interview, which he later apologized for (not that it prevented him from losing his job). Hanson is asking for people to actually stand up for a proposition.

http://www.gwern.net/ gwern

> Watson made an off-the-cuff response in an interview, which he later
apologized for (not that it prevented him from losing his job)

That would seem to increase the force of my example, not decrease it.

http://entitledtoanopinion.wordpress.com TGGP

True enough. Robin is asking for something more than Watson did.

WT

Note that the award went primarily (in so far as this can be said about a shared award) to Shiller for statistically demonstrating the strong limits of the efficient markets hypothesis. Fama got the award for his contributions in the field (basically creating it), but Shiller got the award for being the one in the field who was correct.
Efficient markets are not generally that maligned, they are extremal popular among Thomas Friedman level columnists who like to think they are smarter than subject matter experts.

http://misunderstoodfinance.blogspot.com/ Milton Recht

For example, TIPS forecasts inflation. TIPS also allows for a more accurate estimate of the real rate of return. The real rate of return forecasted using TIPS is about half the real rate estimated prior to its existence, around 2 percent from TIPS versus about 4 percent commonly stated prior to TIPS first issuance.

US could issue a bond that would aid in the forecasting of future real GDP. Various bond forms whose value would change based on GDP expectations have been discussed by M. Friedman, R. Shiller and others. Would remove a lot of guesswork from the Fed and could be used to set effective long-term economic growth policies.

To those who are confused by short-term unpredictability with long-term predictability consider a stripped interest (principal only) discounted government bond. Today, one can buy a 10-year US bond that is stripped of interest payments and will pay the full principal of $1000 per bond at the end of the 10 years for about $780 today.

At the end of 10 years, the buyer will get $1000. In the 10-year interval between purchase and redemption, 2012 to 2022, as interests rates randomly move up and down, the market value of the bond will fluctuate. If one holds until maturity, one gets $1000. If the investor sells prior to that date, the investor can get more or less than he/she paid to purchase the bond and also more or less than the final principal value of $1000.

The buyers know the final value of the US bond but does not know the intermediate value if he/she has to sell prior to maturity.

IMASBA

“US could issue a bond that would aid in the forecasting of future real GDP. Various bond forms whose value would change based on GDP expectations have been discussed by M. Friedman, R. Shiller and others. Would remove a lot of guesswork from the Fed and could be used to set effective long-term economic growth policies.”

Who cares if China or the Koch Brothers decide to “massage” the price of such a bond, after all, what could possibly go wrong?

Hannes

Massage the price away from a truthful forecast is quite expensive (given that we don’t live in a world where expected GDP growth has a very strong causal effect on realized GDP). I would worry more on massaging taking place on the measurement side.

http://juridicalcoherence.blogspot.com/ Stephen Diamond

Is that because they don’t notice, or don’t care, or what?

You should ask them.

Robert Koslover

I agree with you. I’m guessing that these new Nobelists are: (1) quite possibly aware of at least some of Robin’s well-regarded work concerning prediction markets, and (2) take that work seriously enough that at least one of them would be happy to reply, if contacted.

vaniver

I get the sense that the week someone gets the Nobel Prize is a spectacularly bad time to contact them, if you want a response.

Robert Koslover

No doubt. So wait a few months!

Doug

“Is that because they don’t notice, or don’t care, or what?”

There are many deep-pocketed and powerful institutions that have a deep vested interest in extending speculative markets to cover new areas. For example the Chicago Mercantile Exchange, is the most valuable exchange in the world, essentially self-regulating and free to set up many speculative contracts under its mandate as a derivatives exchange.

And indeed it can and frequently does launch new products designed to make markets in a wide range. Weather derivatives are a big one, like predicting the forecast temperature or precipitation. Real estate is another big one, predicting future aggregate house prices or home sales.

And the thing is these markets largely fail to attract hardly any traders. Despite the CME constantly pushing their best marketing efforts behind them. Mostly because traders aren’t interested in expressive views on an array of carefully tailored contracts. They’d much rather express their views with a few highly liquid instruments that they can easily price, risk manage and trade. The exact translation from view to position is noisily blunted, but the tradeoff is generally worth it.

Believe me, I’m a full-hearted supporter of futarchy. But I’m skeptical that the only barriers to these markets are legal or institutional in nature. Even if it was fully legal and sanctioned to have a market on, e.g. the expectation of US life expectancy in ten years, I doubt it would the liquidity that even a small-cap stock has. And without the liquidity, the efficiency and hence the information content of the market, is quite weak.

Even Intrade which was pretty close to quasi-legal for a good run failed to build anything but microscopic liquidity on any contracts. And that’s only considering top two dozen most popular contracts, most of the contracts on the platform had zero effective liquidity.

http://overcomingbias.com RobinHanson

You are mostly talking about new markets to hedge risks. These Nobel winners have indeed supported such things, have have lots of others in finance. And yes they often fail to attract risk-hedging traders.

I’m talking about markets for the purpose of information aggregation. And for them the customer must be someone who wants to know the answer, and is willing to pay to subsidize such markets to get the answer. And yes even if legalized such markets may be limited due to a limited desires to actually know answers. People say they pay for media, pundits, think tanks, etc. for that purpose, but in fact such things are dominated by other purposes.

citizen15

Doug’s point about attracting liquidity is a good one, and I’m not sure Robin’s answer completely addresses it.

It seems to me, that most of the deep and liquid (and hence most likely to be informationally efficient) financial markets attract participants with non-informational motivations. For example, passive stock investors participate in equity markets to gain equity exposure (“beta”). Commodities producers and consumers may participate in commodities futures markets to hedge risk, as Robin points out. Even sports gambling markets provide entertainment value, which may attract participants that don’t actually believe that they possess superior information.

The presence of non-informed participants may be helpful or even essential in attracting both market-makers to provide liquidity and informed participants. Market makers may believe, that while there are some participants with more information than themselves, there are enough non-informed participants that the market makers can earn enough net profit from the bid-ask spread to compensate for the losses resulting from trading with informed traders. The presence of non-informed traders may help convince informed traders that the information they think they possess truly has not yet been incorporated into market prices. In short, the presence of non-informed traders may help market makers and informed traders answer the question, “Why would I want to trade with someone that is actually willing to trade with me?”

Markets (financial and non-financial) exist to allow trading between parties with *different* needs, preferences, and characteristics. In a pure information market, what asymmetry between market participants would induce trading? Without such asymmetry, it seems like we would have to rely on participants’ overconfidence in their own information. What are the most liquid pure information markets that have been established to date?

If an information market cannot attract non-informed participants, then the subsidy that Robin’s hypothetical customer would have to pay might be quite higher than the cost of setting up an exchange. The customer might need to act as a market-maker, providing liquidity only to informed traders, which could result in substantial trading losses, much higher than the cost of media, pundits, etc. I agree, though, that in many cases those alternatives are established for reasons other than information gathering.

citizen15

Upon further thought, an alternative to customer-as-market-maker might be for the customer to stake a bunch of pundits to trade with each other instead of pontificating about their views. Winning pundits would get a share of their profits, and losing pundits’ losses would be borne by customer. That might provide enough non-informed participants willing to trade (because they’re betting with someone else’s capital) to attract informed participants, even outsiders betting their own capital. The profit opportunity for the informed participants would still be limited by customer’s willingness to lose money in trading. However, at least the pundits would be incentivized to be “right” about their expressed views.

VV

What stock markets are “trying to predict” is the future discounted cumulative dividends that the stocks will pay.
Is there any evidence that they are good at predicting that?

IMASBA

There is reason to believe the stock market isn’t as successful as advertised:

1) many a trader’s success can be sufficiently explained through random chance

2) the biggest traders have shown they can game the system (create artificial demand for their own gain)

3) businesses have shown they can mislead traders

4) traders regularly get it wrong on a massive scale (bubbles)

5) businesses get a lot of their investment from consumer spending and technological progress rather than the stock market and public corporations represent only part of GDP, so a growing economy is not evidence of the accurate predictions of the stock market, for all we know the stock market slows down economic growth

6) Fama, Hansen and Shiller developed a model that could predict, less worse than blind guessing, long term trends but this model at certain points actually goes against the stock market index

7) Fama, Hansen and Shiller’s model became obsolete the day they published it if traders incorporate it into their own models, the same fate awaits every publicly known stock market prediction model

http://juridicalcoherence.blogspot.com/ Stephen Diamond

From the link:

He found that stock prices fluctuate much more than corporate dividends, and that the ratio of prices to dividends tends to fall when it is high, and to increase when it is low. This pattern holds not only for stocks, but also for bonds and other assets…

The Laureates have laid the foundation for the current understanding of asset prices. It relies in part on fluctuations in risk and risk attitudes, and in part on behavioral biases and market frictions.

Consider that the non legality of prediction markets as evidence for or against the idea that those who set legal regimes think accurate information would be to their benefit or detriment. Note that if lobbies wanted these markets to exist they would.

Annonomous

I can only speak to Prof. Schiller, who was one of the very first supporters of HedgeStreet back in the day and in fact wrote a letter to regulators in support of the Company some ten years ago. He is deeply knowledgeable of the structure of markets, and their information aggregation properties, so he definitely knows about info aggregation, but for a variety of reasons chooses not to fight this particular cause. Not sure why, can’t really speculate, but in his case it is definitely a case of being aware of these properties and for some reason no deciding to focus on advocating them.

http://juridicalcoherence.blogspot.com/ Stephen Diamond

If this summary is accurate, Prof. Fama would be the one more likely to support prediction markets:

No, it’s a lot more complicated than that. Peter Hansen made techniques that measure the performance of models, so he’s already out of the pircure. Fama said market prices behave according to random walk and Shiller discovered a pattern in some deviations from this random walk (bubbles, investor panic, billionaires betting against the market, etc…) What this means is that Fama believes all the information available to investors in already incorporated into the market price, but it may very well be that all the information available to the investors is not sufficient to reliably predict a future event that is primarily driven by non-market forces (such as a political election). Shiller believes investors, as a group, will not act entirely rational upon the information that is available to them and that just adds to the problem, especially since the dynamics of primarily non-market driven events are often very different from that of market driven events (for example a political hype can actually help a political candidate indefinitely, whereas a market bubble always eventually ends in a bust).

If Fama and Shiller believe that information available to investors is not enough to reliably predict primarily non-market driven events then they will not support futarchy.

http://juridicalcoherence.blogspot.com/ Stephen Diamond

Reliability isn’t the right concept here; it’s accuracy. If prices (per Fama) incorporate all the information available, then they do the most accurate job possible in predicting from the available information–prima facie, supporting prediction markets.

IMASBA

“If prices (per Fama) incorporate all the information available, then they do the most accurate job possible in predicting from the available information”

Yes.

“prima facie, supporting prediction markets.”

No, reliability (or accuracy if you prefer) may still be very low because the information required for good predictions is not available to the investors (inside information is required or it’s really complicated or not even feasible to compute). Also Shiller’s findings should not be ignored, especially in smaller markets (small compared to the largest stock exchanges) and with timescales of less than multiple decades.

IMASBA

So it may very well be that a prediction market is no more reliable than tossing a coin or if it is more reliable it would still come with such a low confidence level that it does not yield practical advice besides a long term average, except that interesting events are not common enough to repeat a large number of times before real world circumstances have changed (you might get a prediction that under current real world circumstances the US Democratic party has a 55% probability of winning the next presidential election, but the usefulness of that prediction is almost zero since there are 4 years between US presidential elections and real world circumstances will have changed a lot even 5 elections from now and 5 or less repetitions aren’t really statistically significant at 55-45 odds.)

http://juridicalcoherence.blogspot.com/ Stephen Diamond

No, reliability (or accuracy if you prefer) may still be very low because the information required for good predictions is not available to the investors (inside information is required or it’s really complicated or not even feasible to compute).

The idea is that when the information isn’t feasible to compute, then markets come into their own. The market accomplishes an aggregation no individual computing agent could–exactly because the information is so complex and dispersed.

If there’s information that’s not available to investors, then either it’s available to other people or not. Only in the former case is it a potential argument against prediction markets. If it isn’t available to anyone, then prediction markets (if they correctly combine the available information) will do better than any other method of prediction. And that’s the only relevant question (besides how much do the methods cost to implement, which is the big question that Robin tends to avoid).

IMASBA

“The idea is that when the information isn’t feasible to compute, then markets come into their own. The market accomplishes an aggregation no individual computing agent could–exactly because the information is so complex and dispersed.”

That only works for existing financial markets because the participants each know a different piece of the puzzle: their personal preferred price and personal flexibility (the market solves a communication problem), plus the market is a self-fulfilling prophecy machine. With prediction markets you lose those properties (the event can completely ignore the prediction market and rather than having unique and sufficient pieces of the puzzle the participants all hold similar or even the same piece, because they’ll rely on a handful of complicated calculation models, and even if they had unique pieces there would be no guarantee they were sufficient), the dynamics are just fundamentally different. I don’t quite know why Robin Hanson doesn’t see that.

“If it isn’t available to anyone, then prediction markets (if they correctly combine the available information) will do better than anyother method of prediction.”

The least worst option can still be so bad it’s not worth it (and this is to be expected for most interesting events you’d want to predict) and as you say these prediction markets come with a cost.

Bottom Line

The fundamental problem with prediction markets is that the vast majority of people don’t vaguely understand uncertainty at all, and the very small select few don’t understand it deeply at all.

So prediction markets provide a solution to a problem that in the minds of many doesn’t exist.

That is why they are not used or respected. Bottom line. End of story.

This is a blog on why we believe and do what we do, why we pretend otherwise, how we might do better, and what our descendants might do, if they don't all die.