I Didn’t See That Coming

Why don’t we anticipate large scale events better? Giant hurricanes (again, the 3rd in ten years)…500 year floods (again, the 3rd in Houston in three years by at least one account)… the crash of the economy… the election of crazy people… the list is getting pretty darned long. People’s inability to see the coming tsunami wave is analyzed quite well in a book I recently read: The Black Swan by Nassim Nicholas Taleb.

The human mind suffers from three ailments as it comes into contact with history, what I call the triplet of opacity. They are:
a. The illusion of understanding…
b. The retrospective distortion…
c. The overvaluation of factual information and the handicape of authoritative and learned people
—The Black Swan

Taleb’s book is only ten years old but already a classic. I read it on the mini-bus driving around the quiet hills of Ireland, and I can’t imagine a better way to absorb such an indictment of our human myopia. It’s very readable; there are some numbers in it, but mostly in the footnotes or the appendix. Most of it is anecdotes and stories, which is kind of ironic, since one of Taleb’s main points is that we rely on anecdotes to understand things because we can’t cope with the math. As it turns out, that’s probably okay, because we aren’t using the math properly anyway.

The Illusion of Understanding–Don’t Be the TurkeyOne way Taleb says we fail to predict properly is in our inability to understand the world in front of us. The world is complicated and large; it’s hard to take it all in. As a result, we either (a) conclude that we can’t predict anything because it’s too complicated or (b) we rely on simply models and create quasi-statistical understandings entirely based on the present. These models fall apart if what our scope is limited. The best example of this is Taleb’s Turkey analogy.

The turkey, born on January 1st, for example, learns to look forward to the chef. The chef feeds him every day, lovingly popping the tastiest grains and morsels into his little mouth. For 330 days, he sees that chef come over and knows, from experience, that something good’s gonna happen.

Until it doesn’t.

Tom sez: Is “gobble gobble” something I’m supposed to repeat or are you thinking of something else?

Like the turkey, people use what they know of the ordinary to predict the extroardinary. Nothing bad will happen if I build my house next to the beach because so many other houses are there. Somebody would have told me if eating this was unsafe. It must be true because I read it on the Internet. The solution for this is a mix of common sense and planning for downside risk. The world isn’t full of altruists, especially when it comes to “easy, no-risk” weight loss/mortgages/student loans/”make thousands of dollars in the comfort of your own home” plans. Really, we’re more like Pinocchio riding in a bus towards Pleasure Island. Maybe we need to think about getting off the bus.

Retrospective Distortion–History is Written by the Winners
Another problem in Taleb’s “triplet of opacity” — and I say this as someone who loves to read about history — is that history is biased. I don’t mean all of history; all of history happened, right down to the dead peasants, rampant cholera outbreaks, and petty bureaucrats. But history tends to be written by the winners or the ones with resources, so it focuses on anecodotes about the positive and stories that glorify the ones who won. As Taleb says:

…history seems clearer and more organized in history books than in empirical reality…”–The Black Swan

In history books, the past is reduced to a few bullet points. I’ve written plenty of Executive Summaries, so I know how this works. People only have so much time to understand what happened, so some of the messy and confusing bits get left out. Drawing conclusions from that becomes faulty.

We also prefer the story to the details. A history book that tells about the background of a famous leader is more interesting than one that talks about the detailed lives of the millions of anonymous poor. So we under and over-estimate causes and effects, creating predictions based on the examples of Abraham Lincoln or Nelson Mandela.

The historian’s view of Waterloo– Wellington did everythingThe Diary view of Waterloo–utter chaos

One of Taleb’s suggestions to counteract this bias is interesting. He suggests reading diaries rather than biographies. A diary written at the time will help portray what it was really like. Maybe this is why The Diary of Anne Frank is always so compelling. We know what happened to the author, and so reading the point of view of one who doesn’t know what’s coming helps us to understand what it was really like.

Maybe we should also write more diaries, so that our grandchildren and grand-nephews and nieces will better understand what we were thinking. It would be easier for them to perceive how limited our viewpoints were at the time.

Overvaluation of Facts — The Fallacy of the Nerd
The third fault that Taleb identifies is the misunderstanding and misuse of experts. Twenty years ago, it was commonly thought that if only we could put the world’s knowledge in front of people, this would solve all world problems. Once we had data at our fingertips, we’d do the right thing. We know now how that turned out. More information available doesn’t lead to better conclusions; it seems to make them worse. Google search results become six pages long, and many of the suggestions lead to false paths. Having access to more experts hasn’t helped either. Experts don’t seem any better than the average person at making predictions. There are several reasons for this.

In some cases, the data they’re using is wrong or limited–remember when the NASA satellite slammed into Mars? Scientists infamously used meters rather than feet when estimating the speed of the approaching body. In other cases, data is analyzed over an inappropriate time period. Any climate change graph that looks only at the last ten years or compares or different times of the year isn’t going to create a useful conclusion.

Also, sometimes the experts disagree. This often happens with reported study results. As far as I can remember, studies have shown that caffeine causes tumors, heart disease, and, most recently, longer life. Which is it? Saturated fat is bad, as was revealed from thousands of studies, until new studies report that it’s not bad. So, pile on the butter, yes?

Most commonly of all, experts draw the wrong conclusions from their data, and they do it big time. In this social media age, information is reduced to a simplistic conclusion or a headline. Go look at polls from November 7, 2016 as examples. You can read them today; Candidate X had a 90% chance of winning because they were ahead by Z percentage points. Forget that statistics aren’t meant to be used that way. Forget that the margin of error was bigger than the gap between candidates or that the gap was 3% between two candidates but that 16% of the population declined to answer. Even today, you can go and look at the websites from November 7 and see so much information devoted to supporting that conclusion, as if having more details about those polls would force the conclusion to come true. Particularly since, at the end of the day, we shouldn’t care about polls, period. Not to tell us what to do. You should vote; I should vote.

If we can’t trust experts or statistics or if we have myopia looking at history, then what should we do? How should we anticipate; how should we predict?

1. Focus on the Big Picture. Instead of relying on a study reported yesterday, what does the data say as a whole? Look over long periods of time or if you have to, look at a picture. Compare Mount Everest today to a picture from 1950. Draw your own conclusions.

2. Be wary of conclusions drawn based on polling or sampling. Those are useful if you work with data, if you’re a scientist, or if you know the limitations of statistics. The average Jane like me has no business making decisions based on polling data – decisions like should I vote, should I start eating a lot of butter again, or should I buy this stock?

3. Taleb has some choice words about planning. He points out the biases in long-term planning. It’s not that we shouldn’t think long-term but that errors in forecasts are so high that we can’t use the results to prove there will be no risk. For example, use the weather report to decide whether it’s better to go on a picnic or to a museum tomorrow. Don’t use a weather forecast to decide where to build your house.

4. Read diaries and more science fiction. Many of the things happening to us have been dealt with as scenarios in books. Mother of Storms, by John Barnes, covers giant hurricanes. Star Wars shows how totalitarian governments use fear to gain control.

5. Be prudent but not fearful. Use common sense, but think like a parent. A parent knows not to let a child play with matches or knives. I don’t KNOW that my child will get injured, but I don’t need to see probabilities on it. I also don’t need to scream every time a child is near a knife.

I was teasing my friends in Oregon about how concerned local governments were about the eclipse. There were many dire warnings about the million people coming into the state with fears of all the roads in the state locked up for days. People were told to prepare for bring extra sleeping bags and bottled water in their cars. Yet, although some of those might have seemed exaggerated, the fact is that it was a good idea for law enforcement and medical communities to be prepared. People should take precautions. The worst scenarios didn’t come true in part because some people heeded the warnings and stayed off the roads. Contingency plans are always a good idea. That traffic coming out of Salem was, after all, pretty bad.
Taleb’s book was highly successful, and he’s become something of a minor celebrity. The most common question he gets asked — all the time — is “What’s the next black swan?” Anyone who asks, of course, missed the point of the book.

It probably would be more useful and interesting to read his diary.

Today’s post was entirely inspired the Daily Post word of the day: Anticipate