decision analysis

In their book, Decisive, the Heath brothers write that there are four major villains of decision making.

Narrow framing – we miss alternatives and options because we frame the possibilities narrowly. We don’t see the big picture.

Confirmation bias – we collect and attend to self-serving information that reinforces what we already believe. Conversely, we tend to ignore (or never see) information that contradicts our preconceived notions.

Short-term emotion – we get wrapped up in the dynamics of the moment and make premature commitments.

Overconfidence – we think we have more control over the future than we really do.

A recent article in the McKinsey Quarterly notes that many “bad choices” in business result not just from bad luck but also from “cognitive and behavioral biases”. The authors argue that executives fall prey to their own biases and may not recognize when “debiasing” techniques need to be applied. In other words, executives (just like the rest of us) make faulty assumptions without realizing it.

Though the McKinsey researchers don’t reference the Heath brothers’ book, they focus on two of the four villains: the confirmation bias and overconfidence. They estimate that these two villains are involved in roughly 75 percent of corporate decisions.

The authors quickly summarize a few of the debiasing techniques – premortems, devil’s advocates, scenario planning, war games etc. – and suggest that these are quite appropriate for the big decisions of the corporate world. But what about everyday, bread-and-butter decisions? For these, the authors suggest a quick checklist approach is more appropriate.

The authors provide two checklists, one for each bias. The checklist for confirmation bias asks questions like (slightly modified here):

Have the decision-makers assembled a diverse team?

Have they discussed their proposal with someone who would certainly disagree with it?

Have they considered at least one plausible alternative?

The checklist for overconfidence includes questions like these:

What are the decision’s two most important side effects that might negatively affect its outcome? (This question is asked at three levels of abstraction: 1) inside the company; 2) inside the company’s industry; 3) in the macro-environment).

Answering these questions leads to a matrix that suggests the appropriate course of action. There are four possible outcomes:

Decide – “the process that led to [the] decision appears to have included safeguards against both confirmation bias and overconfidence.”

Reach out – the process has been tested for downside risk but may still be based on overly narrow assumptions. To use the Heath brothers’ terminology, the decision makers should widen their options with techniques like the vanishing option test.

Stress test – the decision process probably overcomes the confirmation bias but may depend on overconfident assumptions. Decision makers need to challenge these assumptions using techniques like premortems and devil’s advocates.

Reconsider – the decision process is open to both the conformation bias and overconfidence. Time to re-boot the process.

The McKinsey article covers much of the same territory covered by the Heath brothers. Still, it provides a handy checklist for recognizing biases and assumptions that often go unnoticed. It helps us bring subconscious biases to conscious attention. In Daniel Kahneman’s terminology, it moves the decision from System 1 to System 2. Now let’s ask the McKinsey researchers to do the same for the two remaining villains: narrow framing and short-term emotion.

In my critical thinking classes, students get a good dose of heuristics and biases and how they affect the quality of our decisions. Daniel Kahneman and Amos Tversky popularized the notion that we should look at how people actually make decisions as opposed to how they should make decisions if they were perfectly rational.

Most of our decision-making heuristics (or rules of thumb) work most of the time but when they go wrong, they do so in predictable and consistent ways. For instance, we’re not naturally good at judging risk. We tend to overestimate the risk of vividly scary events and underestimate the risk of humdrum, everyday problems. If we’re aware of these biases, we can account for them in our thinking and, perhaps, correct them.

Finding that our economic decisions are often irrational rather than rational has created a whole new field, generally known as behavioral economics. The field ties together concepts as diverse as the availability bias, the endowment effect, the confirmation bias, overconfidence, and hedonic adaptation to explain how people actually make decisions. Though it’s called economics, the basis is psychology.

So does this mean that traditional, rational, statistical, academic decision-making is dead? Well, not so fast. According Justin Fox’s article in a recent issue of Harvard Business Review, there are at least three philosophies of decision-making and each has its place.

Fox acknowledges that, “The Kahneman-Tversky heuristics-and-biases approach has the upper hand right now, both in academia and in the public mind.” But that doesn’t mean that it’s the only game in town.

The traditional, rational, tree-structured logic of formal decision analysis hasn’t gone away. Created by Ronald Howard, Howard Raiffa, and Ward Edwards, Fox argues that the classic approach is best suited to making “Big decisions with long investment horizons and reliable data [as in] oil, gas, and pharma.” Fox notes that Chevron is a major practitioner of the art and that Nate Silver, famous for accurately predicting the elections of 2012, was using a Bayesian variant of the basic approach.

And what about non-rational heuristics that actually do work well? Let’s say, for instance, that you want to rationally allocate your retirement savings across N different investment options. Investing evenly in each of the N funds is typically just as good as any other approach. Know as the 1/N approach, it’s a simple heuristic that leads to good results. Similarly, in choosing between two options, selecting the one you’re more familiar with usually creates results that are no worse than any other approach – and does so more quickly and at much lower cost.

Fox calls this the “effective heuristics” approach or, more simply, the gut-feel approach. Fox suggests that this is most effective, “In predictable situations with opportunities for learning, [such as] firefighting, flying, and sports.” When you have plenty of practice in a predictable situation, your intuition can serve you well. In fact, I’d suggest that the famous (or infamous) interception at the goal line in this year’s Super Bowl resulted from exactly this kind of thinking.

And where does the heuristics-and-biases model fit best? According to Fox, it helps us to “Design better institutions, warn ourselves away from dumb mistakes, and better understand the priorities of others.”

So, we have three philosophies of decision-making and each has its place in the sun. I like the heuristics-and-biases approach because I like to understand how people actually behave. Having read Fox, though, I’ll be sure to add more on the other two philosophies in upcoming classes.

In their book Decisive, Chip and Dan Heath write about the need to honor our core priorities when making decisions. They write that “An agonizing decision … is often a sign of a conflict among ‘core priorities’ … [T]hese are priorities that transcend the week or the quarter … [including] long-term goals and aspirations.”

To illustrate their point, the Heath brothers tell the story of Interplast*, the non-profit organization that recruits volunteer surgeons to repair cleft lips throughout the world. Interplast had some ”thorny issues” that caused contentious arguments and internal turmoil.

One seemingly minor issue was whether surgeons could take their families with them as they traveled to remote locations. The argument in favor: The surgeons were volunteering their time and vacations. It seems only fair to allow them to take their families. The argument against: The families distract the surgeon from their work and make it more difficult to train local doctors.

The argument was intense and divisive. Finally, one board member said to another, “You know, the difference between you and me is you believe the customer is the volunteer surgeon and I believe the customer is the patient.”

That simple statement led Interplast to re-examine and clarify its core priorities. Ultimately, Interplast’s executives resolved that the patient is indeed the center of their universe. Once that was clarified, the decision was no longer agonizing – surgeons should not take their families along.

I thought of Interplast as I read the coverage of Brian Williams’ situation at NBC. In much the same way as Interplast, NBC had to clarify its core priorities. The basic question is whom does NBC serve? Is it more loyal to Brian Williams or to its viewing audience?

In normal times, NBC doesn’t have to answer this question. It can support and promote its anchor while also serving its audience. In a crisis, however, NBC is forced to choose. It’s the moment of truth. Does the company support the man in whom it has invested so much? Or does it protect its credibility with the audience?

Ultimately, NBC sought to protect its credibility. I was struck by what Lester Holt said on his first evening on the air: “Now if I may on a personal note say it is an enormously difficult story to report. Brian is a member of our family, but so are you, our viewers and we will work every night to be worthy of your trust.”

Holt’s statement suggests to me that NBC’s core priority is credibility with the audience. I certainly respect that. It also struck me as being very similar to the question Interplast asked itself.

Clarifying your core priorities is never a simple task. Indeed, it may take a crisis to force the issue. But once you complete the task, everything else is simpler. As my father used to say: Decisions are easy when values are clear.

Here are links to two articles from the New York Times that report on how NBC executives reached their decisions regarding Brian Williams. (Click here and here)

*Interplast has been renamed ReSurge International. Its website is here.

The New England Patriots won the Super Bowl because their opponents, the Seattle Seahawks, made a bad decision. That’s what millions of sports fans in the United States believe this week.

But was it a bad decision or merely a bad outcome? We often evaluate whether a decision was good or bad based on the result. ButRonald Howard, the father of decision analysis, says that you need to separate the two. You can only judge whether a decision was good or bad by studying how the decision was made.

Howard writes that the outcome has nothing to do with the quality of the decision process. A good decision can yield a good result or a bad result. Similarly, a bad decision can generate a good or bad outcome. Don’t assume that the decision causes the result. It’s not so simple. Something entirely random or unforeseen can turn a good decision into a bad result or vice versa.

But, as my boss used to say, we only care about results. So why bother to study the decision process? We should study only what counts – the result of the process, not the process itself.

Well, … not so fast. Let’s say we make a decision based entirely on emotion and gut feel. Let’s also assume that things turn out just great. Did we make a good decision? Maybe. Or maybe we just got lucky.

During the penny stock market boom in Denver, I decided to invest $500 in a wildcat oil company whose stock was selling for ten cents a share. In two weeks, the stock tripled to 31 cents per share. I had turned my $500 stake into $1,500. I was a genius! (There’s a touch of confirmation bias here).

What’s wrong with this story? I assumed that I had made a good decision because I got a good outcome. I must be smart. But, really, I was just lucky. And you probably know the rest of the story. Assuming that I was a smart stock picker, I re-invested the $1,500 and – over the next six months – lost it all.

Today, when I evaluate stocks with the aim of buying something, I repeat a little mantra: “I am not a genius. I am not a genius.” It creates a much better decision process.

I was lucky and got a good outcome from a bad decision process. The Seattle Seahawks, on the other hand, got the opposite. From what I’ve read, they had a good process and made a good decision. They got a horrendous result. Even though their fans will vilify the Seahawks’ coaches, I wouldn’t change their decision process.

And that’s the point here. If you want good decisions, study the decision process and ignore the outcome. You’ll get better processes and better decisions. In the long run, that will tilt the odds in your favor. Chance favors the thoughtful decision maker.

What makes for a good decision process? I’ll write more about that in the coming weeks. In the meantime, you might like two video clips from Roch Pararye, a professor at the University of Pennsylvania, who explains why we separate decision processes from decision results. (Click here and here).