Post navigation

Supercrunchers and The Black Swan: Two very different books

How can two so-called “Business books of the year” be so at odds? Supercrunchers, by Ian Ayres, is subtitled “How Anything Can Be Predicted”; The Black Swan, by Nassim Nicholas Taleb, is based on the central idea of our “blindness with respect to randomness, particularly the large deviations”, and the dramatic impact of randomness on our lives and on history itself.

I found Supercrunchers only marginally interesting. That’s perhaps because a significant part of my job and my team’s job is supercrunching: running a large number of controlled experiments with a small or large number of variables, and optimizing for the most desirable outcome. The language of multivariate testing is common enough throughout my organization that it has become second nature to many of us. Where I work, it is not acceptable not to run experiments, or to run them with the wrong control groups. It’s hard for me to disagree with the idea that if you can manage a business through data crunching, it would be a folly not to.

Where Supercrunchers becomes more wide-ranging, in my opinion, is only when it moves from business to public policy: education, justice and medicine are all domains where we ought to seek the truth, and we don’t look at the data often enough. (A notable success of this method: Ernesto Zedilllo’s Progresa program for rural and urban poverty reduction, later renamed Oportunitades. It is a conditional transfer of cash to poor people – get prenatal care, go for nutrition monitoring, keep your kids in school – and it works best if the money is given to mothers, who seem to be more likely than fathers to use it for their children).

The Black Swan is by far the more fascinating book. A black swan, in Taleb’s essay, is an improbable event with the following characteristics: it is an outlier; it carries an extreme impact; and we tend to explain its occurrence after the fact, saying that it was predictable after all. We are blind towards randomness, particularly with respect to large deviations, and we consistently underestimate the probability of black swans, while black swans are all that really matters, in our lives as in social science and history.

Taleb’s book is a very personal rant about an idea that he very deeply cares about, written in a very idiosyncratic style. To understand his argument, consider that any physical or social phenomenon belongs to one of two domains: Taleb calls them Mediocristan and Extremistan. Matters that belong to Mediocristan exhibit mild randomness and are well described by a Gaussian curve: height, weight, car accidents, mortality rates, IQ, incomes of bakers, restaurant owners and orthodontists. Matters that belong to Extremistan exhibit wild randomness and are either described by a very different distribution or totally intractable: wealth, book sales per author, populations of cities, number of speakers per language, earthquake damage, sizes of companies, commodity prices, financial markets. In other words, take a group of 100 random people, then add basketball player Yao Ming to the group: their average height goes up a little bit. But only a little bit, because Yao Ming, as tall as he is, isn’t even twice as tall as the tallest of the 100 random people. Then take another group of 100 random people, add Bill Gates to the mix, and see what happens to their average wealth. That’s because wealth is not normally distributed at all, and Gates is a heck of a lot more times richer than the next richest guy in his group of 100 people than Yao Ming is taller than the next tallest guy in his group. Technology and progress seem to have made the Extremistan-type phenomena more ubiquitous and more relevant than Mediocristan-type phenomena: for example, before the birth of recorded music, income for musicians was a lot more normally distributed than it is today, in what Sherwin Rosen has called “the economics of superstars”.

What are the implications of black swans? Taleb offers few concrete answers, not because he’s a lazy writer, but because to him this is an intrinsic result of his skeptical method. In terms of investment strategy, if you accept that most measures of risk are flawed, then there are really no predictably “medium-risk” assets. You are better off with a “barbell” strategy, putting 85 to 90 percent of your assets into very safe instruments, such as Treasury bills, and the other 10 to 15 percent into venture capital-style portfolios, which are meant to expose you to the good black swan. Other than that, you should try to expose yourself to as many opportunities as you can, which implies living in a big city and going to lots of parties (or, if you are autistic, sending your associates).

The Black Swan is a very seductive book by an intriguing author: one who has decided to “never attend business “meetings”, avoid the company of “achievers” and people in suits who don’t read books, and take a sabbatical year for every three on average.” It does not wear its philosophical ambitions lightly, and it makes Supercrunchers look trivial by comparison. Ayres’s daughter, Anna, may have learned a great deal from her dad in her ability to use the standard deviation concept at the age of eight, but he’s probably doing her a disservice when he implies (in Chapter Eight) that the standard deviation of NYSE returns is 20 percent in a Gaussian distribution – a statement that Taleb, having crunched billions of market data points in his work as a trader, would find entirely nonsensical (random walk, anyone?) The opening case history of Supercrunchers is particularly disappointing: so, some Princeton economist can predict Bordeaux prices from the season’s temperatures and rainfall patterns? Big deal! Anybody can do that – after the weather has occurred. It would start to be interesting if somebody could predict the weather for the 2008 Bordeaux today, in December 2007 – which, as we know, is devilishly difficult. And anyway, if you’re a chateau owner growing your Bordeaux, that’s not what you should worry about. Temperature and rainfall in Bordeaux belong to Mediocristan, so you’re not going to get big surprises one way or the other. What you ought to do is go out and buy insurance against the probability that a volcanic eruption blackens European skies with ash for the entire summer; that a previously undiscovered plant disease wipes out your grapes; that an accident at a nuclear power plant makes your land radioactive for the next three generations. (The mere fact that I am coming up with examples from real events – Indonesian forest fires, phyllossera, Chernobyl – shows you how unimaginative I am in stretching myself to think of real black swan-type risks).

In summary, if you only have time for one book this Christmas holiday, go for The Black Swan. Supercrunchers will get you to optimize your spreadsheets, but The Black Swan will get you to use your brain: and that’s what books are for.

When reviews are this good they make readers lazy. Black Swan was high on my buying list but now, thanks to your selfless assistance, I’ll make do with your post and move on to something else (namely “A Farewell to Alms” – in case you plan to read this one as well, please consider a timely post to slim my list further down). I’ve read 3-4 prime paper reviews on Black Swan: none of them outlined the book kernel as tersely as you did.

By comparing the swan to the crunch, you highlighted a tension which has animated endless debates in philosophies and sciences: random vs deterministic, chance vs necessity, fate vs will. Taleb and Ayres bring this everlong dispute into the realms of finance and business.

One might wonder what makes people lean on either side. I suspect personal history matters, with the lucky typically confident in the human ability to shape events (and thus taking credit for their fortune) and the loser fully aware of the human inability to control circumstances (and thus blaming black swans for their misery).

Call me a loser, but I sympathise with the random side. At the same time, I find hard to ignore determinism’s benefits. It’s not accidental that Taleb has not much to recommend. Milton Friedman famously said that there are infinite ways to be irrational but only one to be rational. That’s why economic and finance models assume rational behaviour: it’s the only way to achieve a limited set of solutions supporting clear recommendations – mostly wrong but occasionally useful.

The Bordeaux example is meant to address the “fallacy” that tasting experts have much better skill at predicting the development of wine than simply tracking the weather during the growth of the grapes. The theme of the book is that experts must fold in available data in prognosticating.

Weather prediction was such a problem decades ago, pitting the Farmer’s Almanac against differential equations and widespread and dense monitoring. The Almanac lost long ago, and the weathermen are gradually extending the range and accuracy of their predictions.

So I think the Bordeaux example is a fine example of recent ascendancy of crunching over the experts.

Just posting because I started Swans last night, and of course both unexpected and predictable events are worth keeping in mind in extrapolation, and our job is to switch as much as possible from the former to the latter.