This combination of low predictability and large impact makes the Black Swann a great puzzle; . . . . . Add to this phenomenon the fact that we tend to act as if it does not exist! I don’t mean just you, your cousin Joey, and me, but almost all “social scientists” who, for over a century, have operated on the false believe that their tools could measure uncertainty. For the applications of the sciences of uncertainty to real-world problems has had ridiculous effect; I have been privileged to see it in finance and economics. Go ask your portfolio manager for his definition of “risk,” and odds are that he will supply you with a measure that excludes the possibility of the Black Swan – hence one that has no better predictive value for assessing the total risks than astrology… This problem is endemic in social matters.

Though published in 2007, we are still living in the aftermath of the impact of the kind of Black Swan he is talking about – an economic meltdown made inevitable by but invisible to those speculators bent on massive profits and convinced of their own infallible prescience.

In his fascinating and meticulously researched book, Thinking Fast, Thinking Slow, Kahneman shows us the blind spots warping our conclusions about the world across many areas of crucial decision-making. The one I’m focusing on right now is finance and speculation on the stock market. I will be returning in a later post to the more mundane aspects which focus on the day-to-day decisions we all make, but I thought this would be a great taster. Most of the material I quote comes from Chapter 20 – The Illusion of Validity.

A Personal Experience of Predictive Failure

The beginning of his interest in predictive judgement in general began with his own experiences as part of a team attempting to predict from a one hour intensive training experience the suitability of military trainees in Israel for officer status. Their methodology was extremely systematic and all members of his assessment team would converge on a confident and united view of each candidate’s prospects at the end of the experience.

However, an examination of the future careers of these recruits strongly suggested that their predictions were only marginally better than chance (my Kindle version unfortunately does not give me page numbers: 3804).

The dismal truth about the quality of our predictions had no effect whatsoever on how we evaluated candidates and very little effect on the confidence we felt in our judgments and predictions about individuals.

When they came to look at the corporate world they found the same pattern of consensus confidence resulting in virtually useless predictions.

Unreliable Prediction in Corporate Life

People generally discount the influence of luck on short-term performance. As a result (3687) ‘[a] few lucky gambles can crown a reckless leader with a halo of prescience.’

The Long-term Overview

When you take a more systematic view of performance over the longer term the picture looks very different (3741-2):

On average, the gap in corporate profitability and stock returns between the outstanding firms and the less successful firms studied in Built to Last shrank to almost nothing in the period following [a highly popular] study. The average profitability of the companies identified in the famous In Search of Excellence dropped sharply as well within a short time. A study of Fortune’s ‘Most Admired Companies’ finds that over a twenty-year period the firms with the worst ratings went on to earn much higher stock returns than the most admired firms.

Kahneman continues (3831):

Since then, my questions about the stock market have hardened into a larger puzzle: a major industry appears to be built largely on an illusion of skill.

He begins his analysis of what goes on in the process of speculation by flagging up the point that for every seller there is a buyer and vice versa, each with their opposite assumptions (3833):

Most of the buyers and sellers know that they have the same information; they exchange the stocks primarily because they have different opinions.

He picks up the paradox (3835):

The puzzle is why buyers and sellers alike think that the current price is wrong. What makes them believe they know more about what the price should be than the market does? For most of them, that belief is an illusion.

Tracking individuals does not confirm their sense that they know what they are doing (3843):

Many individual investors lose consistently by trading, an achievement that a dart-throwing chimp could not match.

Investors, when they looked at the overall picture, clearly expected the stock they chose to buy would do better than the stock they chose to sell. The figures did not confirm that (3850):

Odean [a finance professor at Berkeley] compared [using 10,000 brokerage accounts over a seven year period] the returns of the stock the investor had sold and the stock he had bought in its place, over the course of one year after [each] transaction. The results were unequivocally bad. On average, the shares that individual traders sold did better than those they bought, by a very substantial margin: 3.2 percentage points per year, above and beyond the significant costs of executing the two trades.

Kahneman, in discussing this study, damningly concludes that (3854) ‘it is clear that for the large majority of individual investors, taking a shower and doing nothing would have been a better policy than implementing the ideas that came to their minds.’

His critique of the finance sector in the book as a whole stretches to include Chief Finance Officers as well. His discussion of their weaknesses comes in a later chapter and I will only include a few points here in order not to overcomplicate the picture but simply to show that prediction is a problem in many places.

In Chapter 24, after reviewing the evidence he concludes (4738):

. . . . .financial officers of large corporations had no clue about the short-term future of the stock market; the correlation between their estimates and the true value was slightly less than zero! When they said the market would go down, it was slightly more likely than not that it would go up. These findings are not surprising. The truly bad news is that the CFOs did not appear to know that their forecasts were worthless.

Nassim Taleb

He explains this in terms of the following dynamic (4760):

As Nassim Taleb has argued, inadequate appreciation of the uncertainty of the environment inevitably leads economic agents to take risks they should avoid. However, optimism is highly valued, socially and in the market; people and firms reward the providers of dangerously misleading information more than they reward truth tellers.

This tendency stands on the shifting sand of insecure judgements about complexity in general (4053-4055):

Another reason for the inferiority of expert judgment is that humans are incorrigibly inconsistent in making summary judgments of complex information. When asked to evaluate the same information twice, they frequently give different answers. . . . . Experienced radiologists who evaluate chest X-rays as “normal” or “abnormal” contradict themselves 20% of the time when they see the same picture on separate occasions.

This is a good place to pause before we go on to consider some exceptions to this pattern and also to see the results of Kahneman’s own investigations and his eventual overall conclusions.

One Response

This is fascinating and counter-intuitive (which is in itself part of the fascination). We really do overrate our capacity for making sound judgments and predictions.

And yet some people seem to come out on top. I’m thinking, for example, about the investor Warren Buffet, summarised as follows by Wikipedia: “Warren Edward Buffett is an American business magnate, investor, and philanthropist. He is widely considered the most successful investor of the 20th century.” Now, has Warren Buffet really “won”, or is his success story another example of smoke and mirrors?

We seem unable to avoid the kind of collective delusion that leads to financial and economic bubbles (and the bursting of the same) and to political horrors such as fascism, communism and our present obsession with consumerism.

Good piece, Pete, I look forward to the next post (which, I hope, won’t be the “Last Post”, ho ho!)