Putting the brakes on impact

Social purpose is at the heart of impact investment. A credible and robust marketplace for social impact needs accounting approaches that reflect this central social purpose. The lack of a generally-accepted impact measurement taxonomy is consistently identified as a major barrier to the growth in scale of the social impact investment market. A dominant response to this issue argues for standardisation in metrics of social impact; however this response largely reflects the needs of investors and funders.

This 2010 paper by Ebrahim and Rangan provides a compelling and sensible counterpoint to this dominant argument. It provides a framework for impact measurement that is grounded in the reality of social purpose organisations. The starting point for the analysis is the well-established logic model for social change, as shown below:

The conventional wisdom around this model is that for organisations to show that they are ‘social purpose’ and actually create social impact, they should extend measurement practices as far down the chain as possible. Ebrahim and Rangan argue that this approach is not feasible, or desirable, given the reality of social impact creation. Instead, they propose a framework for measurement that links measurement to the goals and strategies of social purpose organisations.

This contingency framework is based on two dimensions: the complexity of operational strategy and the complexity of theory of change.

The first dimension – theory of change – considers how complex the causal logic is behind an intervention. A focused theory of change refers to interventions where there is a defined and well-understood relationship between cause and effect. An example of this would be emergency care – where providing shelter, food and water meet immediate and basic human needs. Activities with a more complex theory of change includes cause-effect relationships that are only weakly understood, or where effects may be caused by multiple, interdependent causal factors. This may include efforts to change public policy and public opinions, where the results of an intervention may be shaped by a constellation of actors and factors.

Secondly, they consider the complexity of organisational strategy. Focused operational strategies concentrate on a highly specific task or intervention – such as providing ambulance service to transport patients. This task alone can often not be reasonably associated with the achievement of a distinct social goal, but it plays an important part in achieving social change, alongside other interventions. Organisations that provide an integrated series of interventions – such as an integrated suite of healthcare solutions- are instead considered to have a complex operational strategy.

The paper maps these dimensions in a contingency framework, and links these areas to the types of measurement those organisations may reasonably pursue. Some of these findings are summarised in this table:

Operational Strategy

Focused

Complex

Theory of Change

Complex

Institutional Results

Ecosystem Results

Measure outputs and influence

Measure outcomes and impacts

e.g. changing societal norms

e.g. economic development

Focused

Niche Results

Integrated Results

Measure inputs, activities and outputs

Measure aggregate outputs and outcomes

e.g. basic and emergency services

e.g. service delivery

Ebrahim and Rangan use this framework to suggest organisations operating in different quadrants of this model should be trying to measure their impact in different ways. For example, those in the niche quadrant – with a linear theory of change, with a narrowly focused organisational strategy – should focus their measurement efforts on inputs, activities and outputs.

Importantly, this framework suggests that only under the limited set of circumstances known as ecosystem results (complex theory of change; complex organisational strategy) should organisations seek to measure impact. This conclusion is qualified, however, by the concern that the organisation must control the results – that is, it cannot claim results that it is not sure it was responsible for.

Dividing the diversity and complexity of the social purpose sector into four distinct segments is, of course, an oversimplification. The authors acknowledge that organisations may not fit neatly into any of these quadrants, or that activities of any organisation may stretch across multiple areas. Nonetheless, this paper injects rationality into the debate, and encourages deeper consideration of the reality of social purpose organisations and their strategies for achieving social good. This article also intersects with the important issue of proportionality in social impact measurement: that the depth, complexity and resource demands of any social impact measurement should correspond with the depth, complexity and resources of an organisation. This approach places impact measurement in a cohesive conceptual framework, and at the same time allows scope for organisations to choose tools and techniques that suit their own needs and those of their funders. Meeting the needs of both is critical to building an effective and sustainable social marketplace.

n.b. For publicly-available discussion of these issues by the same authors, consider the following:

Knowledge Connect

The Centre for Social Impact can be reached on +61 2 9385 6568 or email csi@unsw.edu.au
We can be found at Level 6, East Lobby, Australian School of Business (E12)
College Road via Gate 2, High Street, Kensington NSW 2033