As a manager, I’ve seen it far too often. It happens at every level with bosses, colleagues, and staff. Someone occupies a new role in an organization, tries to contribute, runs into the bureaucracy, gets frustrated, battles the status quo for a period, loses, and then leaves. Or, worse still, gives up and stays as another tired contributor to the drudgery.

When these people arrive, they are often deemed “change agents” in the organization. This label dooms them from the start. Not because they are mean or cruel or foolish. They are simply poised for a very unfair, practically hopeless fight. Such newcomers are frequently tasked with tackling large problems that are far more systemic than realized, problems that are interconnected to other aspects of the organization and reinforced by other interests that represent opposing values of the system. What appears to be a one-on-one fight between the change agent and the “culture” turns out to be a one-on-one-thousand fight.

For example: until the iPod, Sony was the dominant force in mobile music players with the Walkman. Combined with its entertainment division, which created a terrific catalog of music licenses, the company effectively occupied a nice corner of the broader music entertainment market with devices and content. Sound familiar to Apple?

It was. But the company was organized differently. There were separate divisions assigned to each major component of their market. One division managed the Walkman device’s design. Another oversaw its manufacture. Another managed the music/content generation. Another managed the music licensing. And as I understand it, these divisions didn’t report upward to the same vertical. Music was in the entertainment column. The Walkman was in the electronics column.

By virtue of being assigned to separate divisions with separate verticals, these groups were competitors within the organizational system, each trying to generate their own profits, reduce their own losses, maintain the attention of the executives as “cash cows”, and compete on their existing business models since what brought them to this point of dominance must also be the way to continue it.

This is the system at work.

Ultimately, the system decayed. It didn’t fail. It didn’t die. It simply decayed as the broader market changed rather rapidly in the wake of the iPod’s disruption.

Worse still, Sony saw it coming in many ways. In fact, at one point, as a means of being proactive, Sony had developed three mP3 players with many of the advanced features that the iPod would not develop for many more years. Sony was actually far ahead of the game when it came to the research and product development. So why didn’t they win? Why didn’t they jump to a lead with their digital players?

To some extent, it was status quo bias—why disrupt themselves?

But also, the system (their organization) was battling many issues on many fronts. It’s incredibly hard to be proactive with new devices when you’re having to defend other aspects of the business. The system is designed to protect itself first and promulgate itself second.

So at this time, in the early 2000s, the entire music industry was embroiled in a piracy battle with the advent of Napster, Limewire, and other file-sharing services. Launching a popular mP3 player while the music business struggled against online piracy was akin to tossing kerosene of a garbage fire. The mP3 players would make it all the more convenient for people to play pirated digital music.

A boon, perhaps, for the device divisions at Sony.

A bust for the music and licensing divisions.

That’s how it seemed to the executives, anyway. The warring factions seeking to protect their internal bottom lines argued so much that the heated confusion led to a stalemate. Each group was designed within the system to protect their own profit and loss sheets. No one wanted to cannibalize. So they argued and fought in accordance to these goals and they were successful in doing so.

Meanwhile, a new competitor with no historical baggage, a completely different organizational design, and no divisive factions emerged with a device that was elegant and convenient fully-amenable to the broader consumer desire when coupled with its music service. The iPod and iTunes stole the chance that Sony had to maintain its dominance by being everything that Sony’s existing system (business models, organizational structures) couldn’t be.

When you think about it, however, this was never a fair fight. As written in in Walter Isaacson’s biography of Steve Jobs,

Sony provided a clear counterexample to Apple. It had a consumer electronics division that made sleek products and a music division with beloved artists. But because each division tried to protect its own interests, the company as a whole never got its act together to produce an end-to-end service.

As Jimmy Iovine, a record executive of the time, also said, “How Sony missed this is completely mind-boggling to me. Steve would fire people if the divisions didn’t work together, but Sony’s divisions were at war with one another.”

The original purpose of a hierarchy is always to help its originating subsystem do their jobs better. This is something that both the higher and lower levels of a greatly articulated hierarchy easily forget. Therefore, many systems are not meeting their goals because of malfunctioning hierarchies.

“Malfunctioning hierarchies” is a nice way of describing an organization poorly designed to achieve any goal other than preservation of its position. The warring factions at Sony were a feature, not a bug, and Iovine’s astonishment over the way Sony “missed this completely” is really more about the fact that Sony was the Kodak Film Company of the musical device world. Like Sony, Kodak had once built models of digital cameras (the device that ultimately destroyed their film business) but didn’t pursue those devices for the same reasons—warring factions, multifaceted interests, and contradictory goals.

Just imagine the few loud voices at Sony who tried to convince everyone to build these mP3 devices regardless of cannibalized record sales. These folks had the right idea to keep the innovation going but they were invariably drowned out in the “malfunctioning hierarchy”. They never had a chance. The system wasn’t designed to allow it.

This is the major challenge of large, complex organizations. If I’ve learned anything from studying competitive business strategy, it’s that no one is sitting on the One Great Idea. People arrive at most of these ideas around the same time. Steve Jobs was brilliant but he wasn’t clairvoyant. I practically guarantee that a handful of intrepid (though surely more timid) souls at Sony proposed an idea to make their own “Napster for the masses” and plug it into their proprietary Walkman device.

But what is past is past. Those few advocates at Sony probably don’t talk about these things anymore. Few would believe them. Or, if they did, it would sound like a bunch of sour grapes.

And why is that?

Few among us have any interest in rehashing the past and noodling on the idea of “what might have been.” Our broad societal norms downplay such things. It’s almost as if our social graces and general etiquette were designed to value something else. Not our old ideas or near-misses but our actual accomplishments. The more recent those accomplishments, the better. What have you done for me lately.

There are more intricate aspects to this but, suffice to say, this is all due to the fact that there is yet another system at-play. A system with rules, punishments, and rewards that doesn’t value introspection. Another bad system that will defeat timeless, nuanced insights for the latest, loudest, and simplest. Another “bad” system that defeats “good” things. Every time.