Cracking the Innovation Complexity Code

by Paul Hobcraft

Cracking the complexity code

I’d like to explore further why I feel innovation is a complex adaptive system that needs us all to recognize its make up differently than a ‘simple’ idea management system or pipeline process. It is far more complex than that.

Innovation needs to be managed well.

There was a good article within the McKinsey Quarterly, published in 2007 entitled “Cracking the complexity code” written by three authors Suzanne Heywood, Jessica Spungin and David Turnbull that leads with “one view of complexity holds that it’s largely a bad thing- that simplification generally creates value by removing unnecessary costs”.

Certainly we all yearn for a more simplified life, structure, organization, approach to systems or just reducing complexity in our daily lives to find time for what we view as improving its ‘quality’.

Within the article they argue there are two types of complexity- institutional and individual. The former concerns itself with the interactions within the organization, the latter is the way individuals or managers deal personally with complexity.

The real important take away from this article is when organizations treat complexity as something they must overcome, reduce or try to ignore they miss opportunities. Complexity, the authors argue, should be seen as a challenge to be managed, managed well, and its full potential exploited, not as a problem to be reduced or eliminated. It is through the nature of these complexities we achieve competitive advantage and can exploit more of the flow of knowleldge for those new sources of new profit and wealth creation.

They suggest organizations need to decide on where to hold complexity within any design and build the right capabilities where they matter. I would argue innovation certainly matters, and it is complex and needs to be understood as exactly that, and managed accordingly not in piece meal fashion. Complexity matters in building the right processes, skills and culture but because they don’t behave in linear ways and any ‘messing’ with the complexity and relationships within this can have an awful lot of unintended consequences.

The other correlations that fits for me

The late Everett Rogers offered us the diffusion of innovation, which gave us a frame to understand the process by which innovation spreads within social systems. Complex systems are equally about relationships among the members of a system. You move into more the emergent behaviours that become increasingly adaptive in response to the environment and what interacts within it. Diffusion occurs in complex systems where networks overlap, exchange and learn. Both Diffusion and Complex Systems adapt and adopt with the end point of making ‘it’ into more of an ordered system. The more you work the system, the fitter for purpose it becomes, the more it diffuses out, the more dynamic it becomes and increasingly valuable from these interactions.

Complex adaptive systems don’t operate in equilibrium conditions

I’ve been also in a set of debates in recent days around management looking for stability, for predictably, looking to take as much complexity out of the system as possible- often sometimes labelled as ‘variance’. This leads to enforcing business as usual as the modus operandi for innovation to ‘fit’ but we are faced with the very opposite in today’s world, the need to ‘embrace’ reoccuring change. We need to manage complexity and we do need innovation so we do need to obtain as much diversity and non linear structure in what we do to allow diversity and all possible options. Our innovation systems are being forced ‘open’ making them even more complex and our energies will have to turn from ‘containment’ to more ‘adaptive’ and responsive ones to manage going forward.

We need to not reject complex systems we need to understand them, we need to embrace them and learn to determine what needs to be complex and what doesn’t. This requires a real ‘flow’ of different energies to maintain the organization of the system, it needs active managing. It will only become harder. For innovation to work, to thrive, to provide a sustaining payback, it needs to be seen as a complex adaptive system. We can’t keep hiding and pretending the ‘bits and pieces’ we play with and constantly fiddle with, called our innovation system, will be sufficent. We do need to understand innovation in its entirety.

We have choices of complexity

There are different types of complexity to manage. Work conducted by Julian Birkinshaw and Suzanne Heywood suggested four types of complexity. I only summarize these here. Imposed complexity, those interventions both internally and externally that require ‘higher’ insight. There is the inherent complexity found with any organization and presently managed through striving to be more efficient and effective. There is designed in complexity, where innovation needs to fit more. These are choices about how, where and why an organization sets about its operation. These can be constrained, under invested in, even jettisoned but do have lasting consequences for the future of the organization. This is the area of strategic consequence as these can limit competitive advantaged from the level of innovation intensity chosen as an example. The forth is unnecessary complexity where increased misalignment resides, it is sometimes easy to recognize but often hard to let go as it sometimes makes up “the way things are run around here” and have a richness in history.

The challenge of complexity within innovation

If you can begin to indentify complexity that hampers effectiveness you can begin to remove it but be really clear on the effects if the complexity part you are removing is not the route to value and often innovation, which certainly does seemingly get constrained and caught up in this often shorter term pursuit of effectiveness for effectiveness sake and you don’t have bandwidth for innovation exploriation.

Recognize innovation is complex, recognize it does have to be handled carefully but it needs to also be fully understood for what it is, a complex adaptive system. It cannot be treated in the same way as effectiveness or efficiency can. It needs ‘actively’ managing differently, for all the future opportunities it holds by placing the emphasis on building greater innovation capabilities to make it ‘dynamically’ work. Otherwise you end up with unexplained consequences to poorer performance from your innovation activities and often at a loss to explain why.

We do need to relate more to complexity as it comes with the turf if you want really lasting innovation in my opinion. Do you agree?

Paul Hobcraft runs Agility Innovation, an advisory business that stimulates sound innovation practice, researches topics that relate to innovation for the future, as well as aligning innovation to organizations core capabilities.

No comments

I enjoyed the article but am not convinced it says anything that will make a difference. In my comment which follows this line I am talking of innovation rather than the act of invention. I see innovation as the wider implementation of an invention.

For me, the question is, “Is innovation complex or the way we think of it too simplistic?” When organisations adopt the Design School approach to strategy (e.g. where do you want to be? how will you get there?) then the assumptions about ‘fit’ with external variables remains untested (insular under blind assumptions?). So is the high number of failures telling us innovation is hard or our theories of how to do innovation are flawed? I believe it tells us more about how we think of innovation rather than innovation itself.

There are objective aspects of innovation which can help us think out our theories of innovation such as “Every innovation has more perceived value than the alternatives it displaces”. Yet I don’t think I have seen anyone examine innovation as a theory of technological progress and subject it to critiques in the same way other social science theories might be, let alone attempt to develop a set of axioms to guide practice.

I hope this comment is seen as someone from industry trying to steer the research agenda towards new approaches to innovation. My aim is to start a new debate in which the correspondence between theory and evidence are used to build greater confidence in new theories of innovation.