What business today does not want to be more innovative? In business parlance, “innovation” has reached a glorified position—like “customer centricity,” it is deemed to be a strategic necessity. But it is hard to define. It means different things to different people. Innovation exists along a continuum, from material improvements to existing products or processes all the way to the rare disruptive innovation. For our purposes, let’s define it as a “big new” for your business or a “big different” in how you operate your business.

How does innovation occur? Through an inefficient process of ideation, exploration, and experimentation. The process results in innovation when we co-create something with customers or when we create new value by combining seemingly unrelated things or ideas in new ways, transferring something from one environment to another, or finding new insights in patterns or aberrations. Innovative ideas rarely emerge from an “aha!” moment. Instead, they usually arise from thinking differently than we normally think and from learning.

Innovative thinking, like critical thinking, does not come naturally to most people. That’s one reason innovation is so hard. The past 25 years of research in neuroscience, psychology, behavioral economics, and education have demonstrated that we are highly efficient, fast, reflexive thinkers who seek to confirm what we already know. As Nobel laureate Daniel Kahneman stated, “Laziness is built deep into our nature.” As a result, we are cognitively blind to disconfirming data and challenging ideas. In addition, our thinking is limited by our tendency to rationalize information that contradicts our beliefs and by many cognitive biases. In a nutshell, when we are on autopilot, we are not critical or innovative thinkers—we are confirmation machines. To innovate, people have to take their normal thinking to a much higher level. Most of us have to be taught how to do that.

Thinking differently is also hard emotionally. Many neuroscientists, including Antonio Damasio and Mary Helen Immordino-Yang, believethat our emotions influence and are integrally intertwined in most of our cognitive processing. In other words, rationality is a myth. Emotionally, we seek to affirm our self-image (our ego) and we use the 3Ds—deny, defend, and deflect—to ward off challenges to it and to our views of the world. Fear is one of the emotions that comes all too naturally to most of us—and makes it hard for us to engage in the messy work of innovation. Fear of failure, fear of looking bad, and fear of losing our job if we make mistakes all can lead to what Chris Argyris called “defensive reasoning”: the tendency to defend what we believe. This makes it hard to get outside of ourselves in order to “think out of the box.”

(c)iStock.com/EdHess

Our educational system and most work environments have taught us that good performance means avoiding failure, not making mistakes. This is a big problem, because failure is an unavoidable part of innovation experimentation. Innovation requires the willingness to fail and learn. Abraham Maslow, one of the founders of the humanistic psychology movement, aptly stated that an individual would engage in learning only “to the extent he is not crippled by fear and to the extent he feels safe enough to dare.”

This means that in order to innovate we need to change our attitude toward failures and mistakes. Contrary to what many of us have been taught, avoiding failure is not a sign that we’re smart. Being smart is not about knowing all the answers and performing flawlessly. Being smart is knowing what you don’t know, prioritizing what you need to know, and being very good at finding the best evidence-based answers. Being smart requires you to become comfortable saying, “I don’t know.” It means that you do not identify yourself by your ideas but by whether you are an open-minded, good critical and innovative thinker and learner.

Most organizational environments won’t help us overcome our fear of failure and build our innovative thinking skills. That’s because most organizations exist to produce predictable, reliable, standardized results. In those environments, mistakes and failures are bad. That is a problem. To innovate, you must simultaneously tolerate mistakes and insist on operational excellence. Many businesses struggle with implementing that dual mentality.

Here we can learn from exemplar companies like IDEO, Pixar,
Intuit, W.L. Gore & Associates, and Bridgewater Associates. In those organizations, mistakes and failures are redefined as “learning opportunities.” IDEO takes it even further, characterizing failure as good because it helps people develop the humility that is necessary for empathy—a critical skill in user-centric innovation.

But in many workplaces, people do not “feel safe enough to dare.” They don’t necessarily feel that they can speak with candor up and down the organization. Can you tell your boss the truth? Innovation occurs best in an “idea meritocracy,” a culture where the best evidence-based ideas win. There can’t be two sets of rules—everyone’s ideas must be subject to the same rigorous scrutiny. As Ray Dalio, the founder of Bridgewater Associates, one of the largest hedge funds in the world, so bluntly said, “We all are dumb shits.” That’s why everyone at his company is engaged in a radically transparent “search for truth,” which involves candid feedback and a deliberate effort to “get above yourself,” to get past the emotional defenses that inhibit our thinking.

Intuit spent the past eight years building a culture to better foster experimentation-driven innovation. A key part of that effort was an intense focus on how leaders needed to change their behaviors and thinking. Humility, empathy, and the devaluation of hierarchical rank were critical to making this new culture work. In a November 2012 blog post, president and CEO Brad Smith summed up the shift: “The modern day Caesar is the boss who gives thumbs up or thumbs down on all decisions. Decisions made by politics, persuasion, and PowerPoint. It’s time to bury Caesar.”

Creating a “big new” or a “big different” for your business requires innovative thinking, and innovative thinking requires the right kind of organizational environment. That is why innovation is so hard.

Edward D. Hess is a professor of business administration at the University of Virginia’s Darden School of Business. His 11th book, Learn or Die: Using Science to Build a Leading-Edge Learning Organization, will be published in September.