My favourite was Tim Harford, who provides an eclectic mix of incredibly readable stories and convincing argument. He shows the dangers of sclerosis with a discussion of the US military in Vietnam:

“When the US Army faced the ‘disruptive innovation’ of guerrilla warfare in Vietnam, there was great reluctance to accept that it had changed the nature of the game, making obsolete the Army’s hard-won expertise in industrial warfare. As one senior officer said, ‘I’ll be damned if I permit the United States army, its institutions, its doctrine and its traditions to be destroyed just to win this lousy war.’”

That sets the tone for a riveting account of the random-walk quality of economic history – ‘Few company bosses would care to admit it, but the market fumbles its way to success’. Toyota began as a manufacturer of looms.

He buys into markets-as-evolutionary process (see my rave reviews of Eric Beinhocker’s work on this) and sees ‘the evolutionary mix of small steps and occasional wild gambles as the best possible way to search for solutions.’ He quotes biochemist Leslie Orgel ‘Evolution is cleverer than you are’ and says ‘formal theory won’t get you nearly as far as an incredibly rapid, systematic process of trial and error.’

In this view, the failure of the Soviet Union is down to the death grip of the Planners – ‘its pathological inability to experiment’. Harford is also a big critic of planning on issues like climate change – he sees unintended consequences, loopholes and own goals everywhere. Evolution will automatically attack the rules and find their weaknesses.

So what? ‘Adaptive organizations need to decentralize and become comfortable with the chaos of different local approaches and the awkwardness of dissent from junior staff.’ How to do this? Harford comes up with a ‘Three step recipe for successful adapting: try new things, in the expectation that some will fail; make failure survivable, because it will be common; and make sure that you know when you have failed…… distinguishing success from failure, oddly, can be the hardest task of all’

As an example, he holds up up the Howard Hughes Medical Institute as a model funder of innovation, by funding people not projects and minimising the number of strings attached. Subsequent analysis shows HHMI funding generated research that was important, unusual and influential. Unfortunately it is also a rarity – the world lacks ‘the two elements essential to encourage significant innovation in a complex world: a true openness to risky new ideas, and a willingness to put millions or even billions of dollars at risk.’

Johnson covers some of the same ground in more opaque language, giving particular emphasis to openness and connectivity, which he argues ‘may, in the end, be more valuable to innovation than purely competitive mechanisms… we are often better served by connecting ideas than protecting them.’ He has no time for intellectual property laws that inhibit this connectivity. He ends with some homely, and to my mind rather useful advice:

John Kay’s is a one-idea book from a prolific popular academic and regular FT columnist.Directness doesn’t work – it leads to the kind of modernist architectural horrors of Le Corbusier or the Soviet planning system or Pol Pot; muddling through, lateral thinking and improvisation are better. Notre Dame was built bit by bit over several centuries – no-one started out with a plan.

‘It is hard to overstate the damage done in the recent past by people who thought they knew more about the world than they really did. The managers and financiers who destroyed great businesses in the unsuccessful pursuit of shareholder value. The architects and planners who believed that buildings could be designed from first principles, that vibrant cities could be drawn on a blank sheet of paper, and that expressways should be driven through the hearts of communities. The politicians who believed they could improve public services by the imposition of multiple targets. Acknowledging the complexity of the systems for which they were responsible and the multiple needs of the individuals who operated these systems would have avoided these errors.’

The books coincide in several respects – a rejection of planning; predictability of linear chains of causation as a figment of planner’s imagination and/or a post hoc justification by those who actually followed their instincts and made sense of it afterwards; a celebration of chance and experimentation, of ‘crossing the river by feeling the stones’.

All three books also share a common weakness – they are largely power-free. The evolution of the economy is portrayed as a fair fight between variants (may the best mutant win), rather than taking place on a landscape heavily tilted towards those with power and control, even if they are less fit. That absence of a sense of politics and power may also explain Tim Harford’s fondness of the charter city idea – if it’s an experiment, it must be good.

A lot of this strikes me as highly relevant to development work. They argue that we should shift to a model of experimental pilots + triage that spots the promising ones early on, and kills off the failures (a bit like our accountability programme in Tanzania), but as Steven Johnson argues in his book, we would need to learn to ‘fail faster’ – one of the mantras of the internet startup world. And NGOs just aren’t set up to fail like the private sector is: last word to Harford: ‘The acceptability, even desirability of failure lies at the heart of capitalism – the corporation, limited liability and bankruptcy laws.’

Any further thoughts on their relevance (or otherwise) to our work?

And for those of you who don’t read books, here’s a 3m presentation from Tim Harford, followed by Steven Johnson doing one of those brilliant RSAnimate videos

Well, I’d rather agree that disagree with the authors of the books, though it’s easy to say all these things in theory, much harder would be to imply them in pracise. But experoments should be made, perhaps people really plan too much and it really sometimes prevents from spontanious innovative ideas.

I also wanted to respond to your first question, but perhaps differently. I hear increasing frustration at the worshipping of ‘innovation’. there is a difference between innovation that responds to a new or changed problem and innovation that is innovation for its own sake, to make someone’s name in an academic field for instance. Increasingly I see resources directed to innovation and perhaps away from the less interesting, less ego driven, world of delivery of work we already know works (or so far haven’t confirmed does not work). Your point on power is key. Shifting power relations is potentially a long term grind that probably won’t make you popular or get you noticed. It involves committment over decades to supporting institutions and movements. But if it is what needs to be done, and we are reasonably confident it is important then why so much obsession with ‘innovation’. Obviously in the private sector innovation is more critical when you are seeking a competitive edge, but that isn’t what NGOs care about most…Surely not?

“a rejection of planning; predictability of linear chains of causation as a figment of planner’s imagination and/or a post hoc justification by those who actually followed their instincts and made sense of it afterwards; a celebration of chance and experimentation, of ‘crossing the river by feeling the stones’”

That sounds to me like the antithesis of most NGO management. There’s your relevance.

There is a lot of relevance here. When yo say “Unfortunately it is also a rarity – the world lacks ‘the two elements essential to encourage significant innovation in a complex world: a true openness to risky new ideas, and a willingness to put millions or even billions of dollars at risk.” In the private sector these elements are often driven by Angel Investors (http://en.wikipedia.org/wiki/Angel_investor). Where are the angel investors for the public development sector and how networked are they yet?

This all resonates with what we’ve been finding in the Do No Harm Project. The best field practitioners iterate through their context analysis daily, see where they are having negative impacts, and apply an option immediately. Because they iterate daily, if their option isn’t working or (oops!) is making the situation worse, they know very quickly and they try something else. They look at success the same way: if it’s working, what can I do today to push it!

Thanks for these reviews. All three books have been sitting on my “to read” shelf. I’ll move them to the front of the queue.

If only NGOs could work more like businesses… where it is easier to fail because companies (the good ones at least) see failure as part of the cost of success.

Innovation is especially important in programs focused on institution building and trying to make politics more productive for the poor. Although everyone applauds technological innovation, these areas call out for much more of it.

With the Adapt principles on the one hand, ant the analysis of the Dictator’ s Handbook on the other hand, I have started to look upon proposals differently.

To my regret, I find the Power Impact Analysis of most ” big ideas” as discussed in Busan frightening.

We always knew official development aid can not be expected to be revolutionary, but the balance is more and more tipped to strengthening existing local power structures, and confirming conventional thinking.

Happily, in Humanitarian aid there is more and more focus on innovation and the goal is still on the ultimate beneficiaries and not on all the intermediaries.

Another book to check out is Myths of Innovation, by Steve Berkun. Its an easy, fun read that I found help me really think through innovations I’ve seen or been part of making succeed and fail.

Does touch on power…early on there is a quote from Machiavelli ‘noting more perilous to conduct, or uncertain in its success, than to take the lead in the introduction of a new order of things. For the reformer has enemies in all those who profit by the old order, and only lukewarm defenders in those who would profit..arising from nature of mankind…who do not truly believe in anything new until they have had actual experience of it’ He goes on later to explicity talk about politics and to say ‘hunger, war and poverty are tough problems, but if someone is profiting from these problems, there will be powerful forces encouraging them to continue. Any innovation aimed at solving those problems must consider politics in order for it to suceed.’

I found the book quite practical – really stressed that coming up with the idea is the least of it – its a huge part good problem defintion/analysis, and then an even larger part marketing/selling. Apparently Einstein once said ‘if I had 20 days to solve a problem, I’d spend 19 days defining it’ Gives dozens of examples (edison, ford, wright brothers, etc) of how those we revere for innovations were not the first to solve the problem, they were the first to drive it through to dissemination. He goes through factors of success, and barriers. An important one is ease of adoption – he notes that the cellphone or Internet usage did not climb so fast because its a time of change, but rather because ‘the barriers of entry were low’.

Another interesting link to our work – ‘as a general lesson, large innovations, say, political revolution, bring with them many smaller changes for better or for worse’ (as we’ve learned from emergency response and the opportunties it can provide for addressing inequity).

He ends with a ‘Simple 5 point plan’ for innovation and a great annoitated bibliography. Highly recommended.

About

This is a conversational blog written and maintained by Duncan Green, strategic adviser for Oxfam GB and author of ‘From Poverty to Power’.
This personal reflection is not intended as a comprehensive statement of Oxfam's agreed policies.