Failing by Design

View more from the

Executive Summary

Reprint: R1104E

It’s hardly news that business leaders work in increasingly uncertain environments, where failures are bound to be more common than successes. Yet if you ask executives how well, on a scale of one to 10, their organizations learn from failure, you’ll often get a sheepish “Two—or maybe three” in response. Such organizations are missing a big opportunity: Failure may be inevitable but, if managed well, can be very useful. A certain amount of failure can help you keep your options open, find out what doesn’t work, create the conditions to attract resources and attention, make room for new leaders, and develop intuition and skill.

The key to reaping these benefits is to foster “intelligent failure” throughout your organization. McGrath describes several principles that can help you put intelligent failure to work. You should decide what success and failure would look like before you start a project. Document your initial assumptions, test and revise them as you go, and convert them into knowledge. Fail fast—the longer something takes, the less you’ll learn—and fail cheaply, to contain your downside risk. Limit the number of uncertainties in new projects, and build a culture that tolerates, and sometimes even celebrates, failure. Finally, codify and share what you learn.

These principles won’t give you a means of avoiding all failures down the road—that’s simply not realistic. They will help you use small losses to attain bigger wins over time.

The Idea in Brief

If you’re launching a new business, creating a new product, or developing a new technology, the principles of intelligent failure provide both logic and a safety net.

Decide what you’re trying to do and what success would look like.

Be explicit about the assumptions you’re making and have a plan for testing them throughout the project.

Design the initiative in small chunks so that you learn fast, without spending too much money. Don’t try to learn more than one significant thing at a time.

Create a culture that shares, forgives, and sometimes even celebrates failure.

It’s hardly news that business leaders work in increasingly uncertain environments. Nor will it surprise anyone that under uncertain conditions, failures are more common than successes. And yet, strangely, we don’t design organizations to manage, mitigate, and learn from failures. When I ask executives how effective their organizations are at learning from failure, on a scale of one to 10, I often get a sheepish “Two—or maybe three” in response. As this suggests, most organizations are profoundly biased against failure and make no systematic effort to study it. Executives hide mistakes or pretend they were always part of the master plan. Failures become undiscussable, and people grow so afraid of hurting their career prospects that they eventually stop taking risks.

I’m not going to argue that failure per se is a good thing. Far from it: It can waste money, destroy morale, infuriate customers, damage reputations, harm careers, and sometimes lead to tragedy. But failure is inevitable in uncertain environments, and, if managed well, it can be a very useful thing. Indeed, organizations can’t possibly undertake the risks necessary for innovation and growth if they’re not comfortable with the idea of failing.

What Are Your No-Fail Zones?

Leaders need to be clear about where failure will be tolerated—and where it won’t.

Mike Eskew, the former CEO of UPS, put the customer experience out of bounds: “We fail in such a way that it never touches the customer,” he said. In practice this meant that UPS didn’t experiment with moving, paying for, or otherwise interacting with a package. In all other respects it permitted—even encouraged—entrepreneurial experiments that stretched the century-old company.

An alternative to ignoring failure is to foster “intelligent failure,” a phrase coined by Duke University’s Sim Sitkin in a terrific 1992 Research in Organizational Behavior article titled “Learning Through Failure: The Strategy of Small Losses.” If your organization can adopt the concept of intelligent failure, it will become more agile, better at risk taking, and more adept at organizational learning.

How Failure Can Be Useful

Some of the failures I’m about to describe were the results of intentional experiments. Others were completely unplanned and unexpected. But all of them provide valuable takeaways. A certain amount of failure can help you:

Keep your options open.

As the range of possible outcomes for a course of action expands, the chances of that action’s succeeding diminish. You’ll improve your odds if you make more tries. This is the logic driving businesses that operate in highly uncertain environments, such as venture capital firms (whose success rates range from about 10% to about 20%), pharmaceutical companies (which typically create hundreds of new molecular entities before coming up with one marketable drug), and the movie business (where, according to one study, 1.3% of all films earn 80% of the box office).

Learn what doesn’t work.

Many successful ventures are built on failed projects. Apple’s Macintosh computers emerged in part from the ashes of a now-forgotten product called Lisa, which introduced a number of the graphical user interfaces and mouse operations in today’s computers.

In truly uncertain situations, conventional market research is of little use. If you had asked people in 1990 what they would be willing to pay for an internet search, no one would have known what you were talking about. A massive amount of experimentation was needed before workable search engines emerged. Early entrants sought to be paid for doing the searches themselves. Later, companies explored business models based on advertising. Later still, Google developed a system to maximize the profitability of the ad-based model. Without all that trial and error, it’s highly unlikely that Google could have built the algorithm-based juggernaut so familiar today.

Create the conditions to attract resources and attention.

Organizations tend to move on to new projects rather than fix systemic problems with existing ones. Let something big go wrong, though, and it’s all hands on deck!

I was personally introduced to how failure can be used strategically years ago, when I worked for the City of New York. I ran an IT group charged with installing an automated procurement system. I was blissfully unaware of how challenging it would be to gain political support and financial resources for the project. Luckily, my boss was a political genius. One afternoon, while I was running some analytics, I learned that the data in the old system had become corrupted. I leaped into action, determined to save the day. But after I ran my plan past my boss, he quietly said, “Don’t do any of that. Sometimes things have to fall apart before anybody musters the will to fix them.” He was absolutely right. The failure of the old system created a compelling argument for the new one and was a turning point in gaining support.

Make room for new leaders.

Sad but true: Even today many leadership positions are held by people very much like those who selected them. Entire industries have suffered the consequences of “lifers” who don’t challenge unspoken assumptions and taken-for-granted rules. Only when those assumptions and rules are proven ineffective—often, unfortunately, in the course of great trauma—do boards recruit fresh leaders. The change can be surprisingly beneficial. The U.S. auto industry provides a case in point. Who would have thought that Alan Mulally, a former senior executive at Boeing, would be an inspirational turnaround CEO for Ford?

Develop intuition and skill.

Researchers say that what people think of as intuition is, at its heart, highly developed pattern recognition. Those who have never faced a negative outcome have a critical gap in the body of experience that intuition is based on. Many venture capitalists won’t invest in a new enterprise if the founder has never undergone failure.

Many venture capitalists won’t invest in a new enterprise if the founder has never undergone failure.

Microsoft’s successful entrant in the game business, the Xbox 360, was developed by a team that had worked on 3DO’s failed game console, the unsuccessful WebTV, Apple’s problematic video card business, and Microsoft’s own short-lived UltimateTV. Having been through so many disappointments, the team members were able to spot warning signs and make smart course corrections. For example, the earlier Xbox had used expensive chips from outside manufacturers, and it reportedly lost about $4 billion from 2001 to 2005. The Xbox 360 team chose different manufacturers, worked in close partnership with them to develop the chips, and retained intellectual property rights to the chips, allowing the system to generate profits very early on.

Putting Intelligent Failure to Work

Obviously, not all failures are useful, and even some that we could learn from should be avoided at all costs. But if you accept that failures will sometimes occur in uncertain environments, it makes sense to plan for, manage, and learn from them—and in many cases to consider them experiments rather than failures. Here are seven principles that can help your organization leverage learning from failure.

Principle 1: Decide what success and failure would look likebefore you launch an initiative.

It never ceases to amaze me how often people working on the same project have entirely different views of what would constitute success. In one case I studied, an organization that made environmental remediation equipment was hoping to introduce a new product line. The marketing group thought the equipment’s selling point was that it met a tough new regulatory standard. The engineering group thought the point was cost-effectiveness—and to keep costs down, it was designing out the very features the marketing group wanted to sell. This gap in understanding could easily have led to a failure of the unintelligent variety. But the company found out about it in time to get everyone on the same page and prevent what could have been a marketplace disaster.

Principle 2: Convert assumptions into knowledge.

When you’re tackling a fundamentally uncertain task, your initial assumptions are almost certain to be incorrect. Often the only way to arrive at better ones is to try things out. But you shouldn’t start experimenting until you’ve made your assumptions explicit. Write them down and share them with your team. Then make sure that you and your team are open to revising them as new information comes in. The risk is that we all have a tendency to gravitate toward information that confirms what we already believe—it’s called confirmation bias. A practical way to address this bias is to empower one of your team members to seek out information that suggests your course of action is flawed. You want to find disconfirming information early, before you’ve made extensive commitments and become resistant to changing your mind.

Organizations that don’t record their assumptions tend to run into two big problems. First, assumptions become converted into facts in people’s minds. During a meeting, a manager might venture a guess that a given market could generate $5 million in sales—and before the meeting ends, the $5 million is baked into next year’s budget! This sort of leap causes all kinds of dysfunctional behavior when the guess, almost inevitably, turns out to be wrong. Second, such organizations don’t learn as much as they could. They may right their course as they proceed, learning as they go, but if they’re not rigorous about comparing results with expectations, the lessons won’t be explicit and shared, and future projects won’t benefit from them.

Having spelled out and revised your assumptions, you should then design the organizational equivalent of an experiment to test them. As with a scientific experiment, the idea is that whether or not the outcome is what you’d hoped for, at least you will have learned something.

Principle 3: Be quick about it—fail fast.

Quick, decisive failures have a number of important benefits. First, they can save you from throwing additional resources at a losing proposition. Second, it’s much easier to establish cause and effect when actions and outcomes are close together in time. Third, the sooner you can rule out a given course of action, the faster you will move toward your goal. And finally, an early failure lessens the pressure to continue with the project regardless, because your investment in it is not large.

A practical way to help ensure that any failure happens quickly is to test elements of your project early on. This is the main reason that “agile software development” often produces better results than the more conventional sequential process of systems design. In an agile environment, small chunks of code are written and shared in a quick, iterative fashion with other programmers and users before the team moves on. This is in sharp contrast to the approach in which analysts spend months documenting user requirements before submitting those requirements to programmers, who only then begin coding. By the time a problem is discovered, a project could have been heading in the wrong direction for years.

Speed may require changing how you allocate resources. Instead of going for maximum NPV over a project’s lifetime, for example, you may want to break the financial evaluation into smaller chunks in terms of both money and time. You may also want to invest in more-flexible assets and people until you have learned enough to confidently build a significant operation.

And the human benefits of failing fast should not be overlooked. If people feel that a project’s failure will doom them to months of waiting for another project, or to losing their jobs, then failure is demoralizing. But if lots is going on and the conclusion of one effort means that they’ll immediately get put on another (possibly more interesting) project, then endings can be positive. At the technical consultancy Sagentia, for example, employees are quick to move from project to project. The finance director, Neil Elton, told me, “They’ll proactively send around e-mails with a mini CV, saying, ‘I was going to be busy, now I’m not. Can you use my skills?’” This attitude is symptomatic of an organization that knows how to experiment intelligently.

Principle 4: Contain the downside risk—fail cheaply.

This is an important corollary to failing fast. Initiatives should be designed to make the consequences of failure modest. Sometimes it’s valuable to test a small-scale prototype before making a significant investment. When the Japanese cosmetics firm Kao was considering going into the manufacture of floppy disks, a big question was whether or not customers would buy Kao-branded disks. So the company went to another manufacturer and bought disks that met its quality standards, put the Kao label on them, and offered them to customers. The response was positive, so the plan moved forward. Had the response been negative, Kao could have stopped the project without incurring substantial costs.

This approach may require breaking ingrained habits. The chief innovation officer of a highly technical company I worked with observed that the company would typically get “some guy in a white lab coat” to do a technical feasibility study before deciding whether to enter a new product area. Such studies are not only expensive—upward of $200,000—but also relatively unindicative of business feasibility. So the innovation officer started making mock-ups of potential new products and showing them to prospective customers. In many instances the company learned that nontechnical issues, such as form factor, usability, and fit with existing systems, would have prevented customers from adopting a product. The difference in cost between the approaches was an order of magnitude: A typical mock-up cost around $20,000. The difference in speed was also considerable: a few weeks rather than nine to 12 months.

3M’s reputation for being failure tolerant took a beating under former CEO Jim McNerney, a GE-trained leader who sought to utilize Six Sigma quality practices throughout the company, even in its research labs. Although these worked wonders in 3M’s factories, the emphasis on generating predictable results hampered employees’ willingness to take risks on unproven ideas. When George Buckley took the reins as CEO, in 2005, part of his challenge was to restore the culture of risk taking. He discontinued the use of Six Sigma in the labs and spurred scientists and researchers to pursue new ideas—provided that the downside was small. During the recession, 3M’s historical philosophy of “make a little, sell a little” when introducing a new product was successfully coupled with Buckley’s emphasis on bottom-of-the-pyramid innovations—inexpensive items that could appeal to very broad markets.

Principle 5: Limit the uncertainty.

There isn’t much point to encouraging failure in an arena your organization is already familiar with. But experiencing it in an arena completely divorced from your current capabilities won’t do you much good either: You probably won’t be able to use what you find out, because you won’t understand the context and you won’t know how to connect what you’ve learned to your existing knowledge base.

Google, which is ordinarily very good at experimentation, went too far afield when it tried to launch a non-internet radio venture. The company wanted to automate the pricing of radio ads, as it had with internet ads. Radio stations would give Google a portion (ideally all) of their ad inventory, and Google would pit advertisers against one another to bid for the spots. Problems emerged, however, because stations were reluctant to give over control. Worse, the Google ads went for less than those sold directly by the stations, and although Google argued that increased demand would eventually drive up the auction prices, stations were unwilling to take the chance. Media buyers, for their part, were reluctant to engage with Google, which refused to continue the conventional practices of negotiating prices ahead of time and bundling ads together. After shuttering the business, in 2009, CEO Eric Schmidt attributed its failure to the company’s inability to measure an ad’s performance on the radio—something it could do on the web by tracking views and clicks. The venture cost the company well over $100 million. That’s not a lot of money in Google’s world; the more important point is that relatively little useful learning seems to have occurred. The chasm between Google’s core business and the radio business proved just too great.

It’s wise to minimize the number of uncertainties that need to be resolved at any particular decision point. One way to do this is through what Chris Zook, of Bain, calls adjacencies. For example, you can introduce an existing product in a new market: IKEA sells essentially the same furniture in many different countries. You can offer your customers a new but related product: Wells Fargo has had a lot of success cross-selling. Or you can build a new business on the foundation of an existing capability: Air Products and Chemicals has done this with its plant management capabilities. The point is to learn from failure (and leverage success) in areas that are fairly close to your established activities. Zook says that the number of major uncertainties should be exactly one. That’s a little extreme. I suggest limiting major uncertainties to those that relate either to the market (pricing, acceptance, form factor, and so forth) or to technology and capability issues (standards, scalability, availability of talent, and so forth)—not taking on uncertainties in both dimensions at once.

Another way to experiment without going too far afield is to break a long-term project up into smaller pieces. Consider the commercialization of nanotechnology: Eventually we’ll be able to construct objects at the level of individual molecules, which will be a truly revolutionary change. But that future is likely to be a long time coming. So for the time being, how are we using nanotechnology? Think wrinkle-free Dockers pants. Think cell phone displays that don’t show fingerprints. Those more modest projects make a lot of sense: They apply brand-new technology to familiar products, which fosters learning.

Principle 6: Build a culture that celebrates intelligent failure.

People often fear that their career prospects will be in trouble if something goes wrong on their watch. (And, of course, they’re often right!) Senior managers need to create a climate that encourages intelligent risk taking and doesn’t punish any failures that result. Some companies have found it useful to codify this principle. (See the sidebar “We Won’t Punish Failure.”)

We Won’t Punish Failure

Here are one UK-based global retailer’s formalized rules for when failure is acceptable.

The effort involves genuine uncertainty.

The outcome will be decisive, because we planned carefully.

It’s riskier to do nothing—or to conduct further analysis—than to act and fail.

The cost is small.

The major underlying assumptions are documented in writing.

There is a plan to test the assumptions.

The risks of failing are understood and, to the extent possible, mitigated.

The cost is contained.

Commitments are scaled according to our increasing understanding.

We’ve defined what success would look like—and the opportunity is significant.

This is an area where CEOs can show strong leadership. A.G. Lafley made fearlessness in the face of failure a core tenet of his time at Procter & Gamble. He said repeatedly that a very high success rate is a sign of incremental innovation, and that he was looking for breakthroughs instead. In his book The Game-Changer, published while he was still CEO, he lists and even celebrates his 11 most expensive product failures, focusing on what the company learned from each. The reasons he gives for the failures range from “required significant consumer habit change” (an at-home dry-cleaning kit) to “small idea” (several new laundry detergents).

During performance reviews one division leader would say, “Show me your scrap heap.” All high achievers try some things that don’t work out.

That kind of culture building should happen at all levels of the organization. One senior division head I worked with would say to his team members during their performance reviews, “Show me your scrap heap.” The request perfectly conveys the idea that high achievers will, of necessity, try some things that don’t work out.

Principle 7: Codify and share what you learn.

An intelligent failure whose lessons are not shared is worth far less than one that teaches something to the group or, ideally, the whole organization. There are many ways to capture and transfer learning. Among the most popular are mini postmortems as a project proceeds, checkpoint reviews as key thresholds are reached, and after-action review meetings at the project’s conclusion. In each case the point is to identify what the assumptions were going in, what happened, what that implies for those assumptions, and what should be done next. It is critical to avoid finger-pointing—restraint that is easier to exercise when the underlying ideas are labeled “assumptions” rather than “projections” or “data.”

I recently facilitated a postmortem for a large organization struggling with an IT implementation that had gone dreadfully wrong. Before we convened, I interviewed key decision makers and developed a time line showing when critical decisions had been made. We kicked off the meeting with some general observations about why IT systems often go awry; the message was “You are not alone.” Next we discussed the core assumptions that had been in place when the project was authorized, some four years earlier; these came as a surprise to the newer members of the team. We then walked through five decisions that had made a big impact on the project’s evolution, discussing the assumptions that were held at the time, what we would have done differently, and what had been learned. The day ended with two breakout sessions: one to determine what to do about the current situation, and one to crystallize lessons that could be valuable in other projects and help avoid similar problems in the future. To make sure the learning was transferred, we charged specific individuals with documenting and communicating those lessons.

Let’s come back to the point I made at the outset: In an uncertain and volatile world, avoiding failure is not an option. If you accept this premise, the choice before you is simple: Continue to use practices that limit what you can gain from failures—or embrace the concept of intelligent failure, in which learning can create substantial value.

How to Walk Away from a Project Intelligently

Even companies with a highly disciplined process for beginning new projects seldom have a good one for getting out. A solid disengagement process includes these steps:

1. Decide in advance on periodic checkpoints for determining whether to continue.

2. Evaluate the project’s upside against the current estimated cost of continuing. If it no longer appears that the project will deliver the returns anticipated at the outset, it may be time to stop.

3. Compare the project with other candidate projects that need resources. If this one looks less attractive than they do, it may be time to stop.

4. Assess whether the project team may be falling prey to escalation pressures.

5. Involve an objective, informed outsider in the decision about whether to continue, instead of leaving it up to project team members.

6. If the decision is made to stop, spell out the reasons clearly.

7. Think through how capabilities and assets developed during the course of the project might be recouped.

8. Identify all who will be affected by the project’s termination; draw up a plan to address disappointment or damage they might suffer.

9. Use a symbolic event—a wake, a play, a memorial—to give people closure.

10. Make sure that the people involved get a new, equally interesting opportunity.

The example set by senior management is crucial. Leaders must be willing to talk about failures and what was learned from them. I’ve seen organizations use symbolic rituals to celebrate a failure that taught important lessons; this can create an environment in which failures are discussable. Making the ground rules for risk taking explicit, whether in a contract or by other means, can be useful as well. Telling stories about failures past can make people more comfortable talking about failures in progress. And having graceful ways to shut down initiatives and move on makes the inevitable failures much more palatable. Fumbling toward success by learning from failure will differentiate firms that can thrive during uncertainty from those that cannot.

A version of this article appeared in the April 2011 issue of Harvard Business Review.