The Growing Anger of the Merely, Barely Middle-Class. Guest post by Sina Odugbemi

Should ODI bite the open access bullet for its journals? Response to last week’s rant on the Academic Spring

What is a theory of change and how do we use it?

August 13, 2013

I’m planning to write a paper on this, but thought I’d kick off with a blog and pick your brains for references, suggestions etc. Everyone these days (funders, bosses etc) seems to be demanding a Theory of Change (ToC), although when challenged, many have only the haziest notion of what they mean by it. It’s a great opportunity, but also a risk, if ToCs become so debased that they are no more than logframes on steroids. So in internal conversations, blogs etc I’m gradually fleshing out a description of a ToC. When I ran this past some practical evaluation Oxfamers, they helpfully added a reality check – how to have a ToC conversation with an already existing programme, rather than a blank sheet of paper?

But first the blank sheet of paper. If you’re a regular visitor to this blog, you’ll probably recognize some of this, because it builds on the kinds of questions I ask when trying to understand past change episodes, but throws them forward. Once you’ve decided roughly what you want to work on (and that involves a whole separate piece of analysis), I reckon it’s handy to break down a ToC into four phases, captured in the diagram.

Power Analysis

At their heart, most aspects of development involve a change in power relations (or should do, anyway). Start by asking what is the nature of the redistribution of power involved in the changes we are seeking? Is it primarily about ‘power within’, e.g. helping women acquire the confidence and knowledge to demand their rights; ‘power with’ (collective organization) or ‘power to’ (e.g. supporting CSO advocacy)? There are lots of other ways of slicing the cake, but the point is to have a proper discussion on the nature and (re)distribution of power.

Useful questions include: What is your analysis of the key forces driving/blocking such a change? What economic or political interests are threatened/promoted by the change? Which groups are drivers/blockers/undecided? Is their power formal (eg elected politicians) or informal (traditional leaders, influential individuals)? Is it visible (rules and force) or invisible (in people heads – norms and values) or hidden (behind the scenes influence). Who do the key players listen to (because that may help us decide on our alliance strategy).

Which individuals are likely to play key roles, either as allies or opponents?

Change Hypotheses

On the basis of the power analysis, come up with some hypotheses for how such change might come about. Note that at this point, we are still looking at the process from the outside, rather than discussing our own interventions:

How is the change we are discussing likely to take place? Through existing institutions, or is change likely to require greater disruption, even conflict (e.g. the Arab Spring)?

What alliances (eg between sympathetic officials or politicians, private sector, media, faith leaders or civil society) could drive/block the change?

What would strengthen the good guys and weaken the bad – e.g. research and evidence, pressure from people they listen to (who are they?) or street protest?

Can you foresee any likely ‘critical junctures’: new governments; changes of leadership; election timetables when change is more likely to occur?

Change Strategy

Is that a metaphor or a critical juncture?

Up to now, you have been looking at the issue from the outside. Now it’s time to start thinking about what Oxfam can bring to the party, in the shape of a change strategy (aka a Theory of Action). What should our role be?

As an active player or through supporting local partners? What combination of on-the-ground programming and advocacy might we try?

Can we use Oxfam’s international identity, eg by cross fertilising with experiences from other countries, or international advocacy in support of change?

Can we identify some easy wins, however small, that might inject some momentum into the work?

Are there some identifiable ‘implementation gaps’ we can work on, which are often easier that starting from scratch.

Who are our potential partners – are they ‘usual suspects’ (local civil society organizations and NGOs), ‘unusual suspects’ (private sector bodies, local/national government, faith leaders) or a mixture of both? What could Oxfam’s contribution be eg helping them develop a clearer theory of change; bringing partners together with other actors to build alliances; building particular aspects of their organizational capacity; research support; funding?

How can we make sure we are prepared to take advantage of critical junctures when they occur (whether anticipated or as surprises)?

Implement and Evaluate

Now let’s assume that (shock!) we may not get it 100% right from the outset. How often do we plan to step back and review what we’re doing and consider a change of direction? Who is empowered to ‘press the red button’ and call a time out for reflection if there is a sudden change of circumstances (political unrest, new opportunity etc)? How are we going to collect evidence to inform these reviews (through impact diaries and other methods)? How do we communicate with the donor that our logframe may be changing?

And then round we go again – in this formulation, ToCs are not so much a tick box exercise, or logframes on steroids, as a way of working, involving a much higher level of ongoing political and power analysis and adaptation.

That’s all fine and dandy if you’re starting with a blank sheet of paper, but the reality is usually that you’re arriving to help teams review programmes that are already under way, with lots of partners, activities and history. Current staff may have inherited the work and not been involved in any original ToC discussion (if there was one). How do we need to adapt the above to make it useful?

One way that emerged from the discussion is a kind of reverse engineering/back-testing.

Take your programme’s current activities, along with the content of the programme documents – initial proposal, assumptions, risks and all the rest, and reconstruct the problems you are trying to address and their underlying causes – there are lots of tools available like problem trees, stakeholder mapping etc

Starting from this, revisit your power analysis and come up with a renewed change hypothesis for what you’re working on – including things that

Early change theorists

are not in the project plan!

Based on that change hypothesis, what kind of change strategies (plural) might make sense. Compare these with your current work – what changes might you want to make to your existing plans and activities and budget allocation to reflect that?

Then go into the evaluate and adapt discussion.

Finally, some common traps to avoid:

Death by diagram: a lot of ToC exercises produce a fiendishly complicated diagram with lots of arrows going in all directions. Drawing the diagram can indeed be a really good way to get a group of people’s heads round a change process. But once you’ve drawn it, throw it away, because it will be a scary barrier to thinking for anyone coming new to the work.

It’s all about us: Yes ToCs are supposed to guide our work, but avoid jumping too quickly into ‘what do we do’, rather than spending time analysing the system (power analysis and change hypothesis). You’re almost always likely to be a minor player in a larger drama, so putting ourselves centre stage makes no sense.

A way to make MEL easier: If anything, a good ToC should make monitoring, evaluation and learning more difficult (but more useful). If you’re just trying to come up with some targets to measure progress, you’re probably on the wrong track (e.g. oversimplifying a system, ignoring critical junctures, exaggerating your importance in the system).

Make sense? All comments, suggestions and links welcome, whether on the theory or (perhaps more important) how to use ToCs in practice.

24 comments

Really interesting and relevant! I just would like to add two points here:

(1) On MEL: I encourage to use MEAL instead, but rather than A for Accountability, use A for Analysis. Especially in dynamic context (ie, fragile states, brewing revolts, etc) constant, continuous and quick analyses ought to be integral

At the end of the day, the theory of change is just that – a theory. Therefore, it needs to be tested (which means strong M&E) all the time. None of those 4 phases are static which means that the ToC has to demonstrate that it is ‘fit to survive’ changes in the environment. Naturally this ‘fitness” is not about strength but more about flexibility and being adaptive.

How often have we seen ToC being developed and then left untouched in the reviews leading to poor outcomes?

The blog is very helpful and on the right track. ToCs should definitely be “a way of working.”

I have a couple of suggestion to your question: “Current staff may have inherited the work and not been involved in any original ToC discussion (if there was one). How do we need to adapt the above to make it useful?”

Just use the current ToC as a starting point for a candid discussion. But you need trust to have a “candid” conversation that challenges assumptions, technical soundness of a proposed change and political strategies etc. (See “Building Effective Teams Isn’t Rocket Science, But It’s Just as Hard” by Douglas Conant in the Harvard blog on April 4, 2012)

So the first order of business is to build TRUST. According to Tway, trust is built from three components: “the capacity for trusting, the perception of competence, and the perception of intentions.”

Another suggestion is to follow your advice of repeating the process (“round we go again”) — only do it as quickly and often as possible. By constantly combining good analysis and political action, technically sound, politically possible answers will emerge, reform conjunctures becomes visible, and change becomes possible.

The ‘logframes on steriods/crack’ point is really the core one here. There’s nothing inherently wrong with logframes, they are much the same as a TOC in terms of setting out the basic structure of whatever the intervention is. It’s why SO MANY theories of change are just sideways logframes with a couple of extra star shapes dotted around them containing something vague about the context.

If we were able to have more sensible conversations about how our interventions were going, we could use logframes perfectly well. The problem is they become ‘lock-frames’ as someone clever said whose name i can’t recall i.e. the conversation becomes about why things aren’t aligning with the plan and what a problem that is.

What we have now, is a situation where you still have logframes which are badly used, ALONGSIDE theories of change which are either exactly the same, or are just tick box exercises to be thereafter ignored. (Yes, there are some good ones too.)

The point of logrames, TOCs and whatever comes next is to get project managers to set out their best assessment of what they think will happen, given what has happened before and on the basis of any analysis that they do. In other words, their theory. A logframe IS a theory of change.

How this conversation changes is a combination of donors being more realistic and open to adaptive management, which is sort of starting, and INGOs and other project managers having some balls and being more assertive about using their position and knowledge to clearly push back and say what is and isn’t possible, which i’ve also occasionally seen.

Duncan, good post. You set a framework in a way that is difficult to argue with it- it contains all that it needs to with maybe one aspect- where does the information that informs power analysis and all subsequent steps come from? What I noticed(and I could be reading it in a wrong way) is that you start talking about the evidence only in the ‘implement and evaluate’ part of your diagram. In my view, if we are to shift the way in which development workers do business, I think the first thing that needs to change is where do we get the information from (in order to do the power analysis) and to what extent does it represent opinions of those who we will work with/for (is it disintermediated) or to what extent does it include biases of project managers, experts, decision makers, etc. If it’s the latter, than the power analysis runs a risk of being a rubber stamp of whatever you want it to be. The question in my view is how to get a continued loop of disintermediated feedback (which then by definition takes the driving seat of the entire process, knocking us out of the centar as you rightly point out) that will then drive your graph –from the power analysis to hypothesis and testing.

Logframes are theories of change – to a certain extent… If the focus is on the process rather than the end product. The advantage of a theory of change is that the end product can look like whatever you want it to look like and use whatever language you like – the danger is that a donor/organisation starts imposing what it should look like – then it starts to inform the process and people start bandying about phrases like ‘means of verification’.
What I find useful in a theory of change process is challenging the language and getting down to the real meaning. So a statement like ‘ women gain confidence and knowledge to demand their rights’ is unpicked – ‘which women?’, what kind of knowledge?’ ‘what does confidence look like?’
It’s also important to map what other actors are doing not just think about them as static – it’s more about ‘what bit of this change can we help bring about?’ Theories of change have a tendency as you say Duncan to focus on what we are doing rather than acknowledging that we are often intervening in a change rather than initiating it.

You might want to look at Prof Malcolm Sparrow’s wonderful book “Regulatory Craft” for a somewhat different take on problem solving. I work in the areas of regulation and governance so the context is somewhat different, but I prefer his methodology to others. He puts a lot of work into the definition of the problem to be solved or issue to be tackled, then hops over solutions and goes straight to ‘what does success look like?’ and how do we measure it? Then he returns to an examination of options for solution, all for reasons that are well described in the book with lots of examples.

Thanks Duncan for the neat summary of ToC. The core is that any ‘theory’ model remains just that unless we adapt the WAY we work, less of an action-orientation and more to an analytic orientation that remains relevent to the ever-changing context of development work. ToC is currently in the ‘tick-box’ category when writing proposals and more internally focused on what we shoould do, rather than a genuine attempt to understand the various contexts in which we operate ,and to relate to those contexts in a meaningful way. Until those that hold the purse strings demand such on-going and meaningful analysis, we will sadly continue to spend a day or so in a proposal writing workshop hammering out our little diagrams that then are never refered to again.

I think it is important to ask what the role of evidence is in Theories of Change. It is unclear whether evidence should be used at all, according to guidance material (see our lit review: http://goo.gl/8GeXLc)

This lack of clarity partly comes down to the purpose of Theories of Change: are they simply uncovered assumptions or are they substantial theories developed from empirical data? For the former, the role of evidence is unclear, however for the latter, the types of evidence used to substantiate a claim and the ways this is collected will be central to validating a Theory of Change.

This links to another concern: what is the role of social science theories of change (the ‘real’ theories of change) in how ‘Theories of Change’ are explored? The questions you suggest asking regarding power analysis and change hypotheses clearly are important questions when considering development interventions. But they are also huge questions! While it’s nice (and probably essential) to get everyone in a room regularly to talk about these issues, it may be the case that this kind of discussion exists in a bit of an organisational bubble. In reality (and I know money for this kind of research doesn’t grow on trees), a full political economy analysis (or other method of analysis) may be required in order to get to grips with some of these questions.

In my work on Sri Lanka, this involved looking at both the reasoning and evidence underpinning The Asia Foundation’s Theories of Change for supporting local mediation practices in Sri Lanka: http://goo.gl/IqluCt

This material may be useful to see how we have approached critically appraising Theories of Change.

Thanks for starting an important discussion. Recently, I have been assisting our own organisation and some small NGOs on this. My suggestion to Duncan is that we must not jump to Power analysis so soon. This is becauase, not all organisations do this, probably not there and it may just be an intention for many. To make a TOC change paper relevant to all, some simple guidance could help.

For example, how to write a TOC change paper from the organisational strategy, how to set a purpose to the TOC development, how it is different from the logframe, why assumptions are important and how and why TOC is different from plans and budgetting procedures. Most important, why and how TOC could be used as a learning tool.

I first wonder if change can be a theory – meaning a concept. Change is a concept in the sense that it has a benchamark – the past – from which we recognise a change. This means that change is defined as a comparison and therefore still linked to the past. If I used to overeat and now I do not overeat anymore, the emphasis on eating has remained, it is only the direction of the emphasis that has changed. I wonder if real change is not in a way the very dropping of a benchmark and therefore a fresh look at things – and therefore no theory at all.

One thing you allude to but could expand on is the role of the theory of change internally. It helps to help shape/form a cross-functional team that doesn’t regularly work together or understand each other well. Additionally, the ToC helps teams create a tool to communicate their work, in the broadest sense with others. It provides the content for the project narrative. This is particularly helpful in large organizations or ones that have staff dispersed all over the world. Also, what would be helpful is to clarify the role of ToC process and product in relation to other processes and products. For me the ToC is the first clarifying step in a much more detailed advocacy campaign strategy development process. The ToC becomes the foundational idea for more tactical plans.

Useful to think of the dissatifaction equation which reminds that change happen if dissatisfaction is higher than potential costs induced by change. and if the change process is credible (sufficient ressources, commitment, competences…) for stakeholders to engage.

You’re a brave man to wade into this one. My experience with TOCs is that they tend to either be very high level (e.g. women are the primary agents of change and through transformative leadership etc.) or down in the weeds (at the level of detailed log frames on the intervention). With clients I tend to work in a two step process. The first is to lay out the full range of factors and actors that might effect an outcome (e.g. more sustainable livelihoods for farmers – everything from trade policy to climate change to credit availaibility, etc) in a kind of barebones way. I then move onto the theory of action of the client on that slice of change they’ve identified as their priority (e.g. credit and market access). I pitch this at a meso-level in terms of being less detailed/activity oriented than a log frame, but more focused and causal chain oriented than the very high level theory of change. I just find practitioners as activists grasp the concept of the theory of action more readily…and I keep the use of the term theory to a minimum as I’ve never found that practitioners eyes light up at the idea of sitting around theorizing.

This can be a simpler 3-step process – assessment (of the situation, be it power, unemployment, malnutrition, etc), analysis (to provide the basis for the selection of change strategy(ies)) and action (to implement the change strategies). Repeating this Triple ‘A’ cycle on an agreed time scale provides the opportunity for learning from implementation – through the second and subsequent rounds of assessment and analysis.

Really interesting blog. I agree with you and many of the posters than grounding your change hypotheses in a particular place and time are critical, hence the analysis part. Whether power, or context, or systems analysis many projects or programmes are launched without enough focus at this level. In terms of developing TOCs CARE and International Alert worked on a joint project looking at how peacebuilding organisations could better articulate (from existing programming) their hypotheses and then go about gathering evidence to support or test their theory. The resulting tools and conclusions are available here for those interested.http://insights.careinternational.org.uk/publications/guidance-for-designing-monitoring-and-evaluating-peacebuilding-projects-using-theories-of-change
While I also agree with many posters that TOC offer a wider variety of purposes, the value of really interrogating assumptions and the ‘hope lines’ expressed by your miracle cartoon remain for me the post practical benefit they bring. For this purpose expressing them as IF X, THEN Y, BECAUSE Z -is a useful format for unpicking assumptions of why or how change will happen.

The log frame analysis has been with development projects for many decades. While various new techniques have augmented the original cause and effect model (TOC including power analysis, systems analysis etc) a key problem is too often overlooked.

That problem is the unintended effects of donor/foundation/CSO requirement that each funded intervention must have its own logframe or TOC that can be used to judge performance of the organization being funded. The result has been fragmentation when there needs to be collective action among numerous players to achieve shared outcomes. (Outcome: Positive change in the quality of life of groups of citizens or positive change in environmental status). At the bilateral and multilateral level, such project based fragmentation is a major cost to national governments. Host governments end up having multiple expensive data bases that die with projects rather contributing to or building sustainable national data base that use a common currency of indicators to allow comparison and tracking of societal and environmental improvement across all investment (private, public, and CSO). Another impact has been a colonization of results based management practice by measurement specialists who produce unwieldy data reports that are not attuned to the needs of decision makers the next level up. Because reports are not used but often checked for reporting compliance only, reporting systems become a ritualistic dance.
Very effective tools for evidence based decision making are being squandered at an enormous cost to development.

Love the comments about death by diagram. As a long standing champion of theories of change in a previous job I have been shocked by how massively unhelpful some of the diagrams that I have seen since I came into the development sector are. I do think that diagrams are helpful but only if accompanied by words that explain them and of the two it is the words that are most helpful.
I think that your focus on power analysis while important slightly misses the point. Surely the key to a good theory of change is a good understanding of the problem that you are addressing. The start of the cycle must surely be a good problem analysis – which should include an analysis of the power dynamics. It is only once you have an understanding (even if flawed) of the problem dynamics that you can hope to change the situation, address the problem and have an impact.
On a slightly separate note I have been referring to both macro and micro theories of change where the macro refers to the generic theories that abound on how development or conflict can be addressed or resolved while the micro refers to the context and project specific logic links within the project plan. I have never been comfortable with this terminology and wonder if we should talk about philosophies of change when discussing the generic and stick with theories of change when getting into the nitty grity of a specific instance. Thoughts?

Thanks for a very good analysis of the need for action after all the theories have been debated upon. I really enjoyed reading this and I see -George Bernard Shaw’s quote “You see things, and you say, “Why?” But I dream things that never were, and I say, “Why not?”