Measuring Social Value

Funders, nonprofit executives, and policymakers are very enthusiastic about measuring social value. Alas, they cannot agree on what it is, let alone how to assess it. Their main obstacle is assuming that social value is objective, fixed, and stable. When people approach social value as subjective, malleable, and variable, they create better metrics to capture it.

Over the last few decades,
many people have attempted
to measure what is sometimes
called social, public, or civic
value—that is, the value that
nongovernmental organizations
(NGOs), social enterprises,
social ventures, and
social programs create.1 The
demand for these metrics has
come from all sectors: Foundations
want to direct their
grants to the most effective
programs; public officials, policymakers,
and government
budget offices have to account
for their spending decisions;
investors want hard data analogous
to measures of profit;
and nonprofits need to demonstrate
their impact to funders,
partners, and beneficiaries.
Metrics to meet these needs
have proliferated over the last
40 years, resulting in hundreds
of competing methods for calculating
social value.2

Despite the enthusiasm for
metrics, few people actually
use them to guide decisions.
In the nonprofit sector, good
managers are very rigorous
about tracking costs and income.
But few use sophisticated
metrics to help allocate
resources. Meanwhile, in the
public sector, political judgment
counts more than cost-benefit
assessments. In the
rare cases when decision makers
do use metrics of social
value, it’s far from clear that
they should.

I’ve dealt with social value
metrics in a variety of roles:
as director of policy and
strategy under United Kingdom
Prime Minister Tony
Blair; as director of the Young
Foundation, an NGO that
has created dozens of ventures,
some for-profit, some
social enterprises, and some
public; and as an advisor to
many other governments. In
these positions, I’ve seen not
only why social value metrics
are ignored, but also how to
make them more useful.

One recent project that
proved particularly informative
was a collaboration between
the United Kingdom’s
National Health Service
(NHS) and the Young Foundation.
The NHS commissioned
the Young Foundation
to develop a practical tool
for assessing service innovations
and guiding investment
decisions. The NHS is a vast
organization with a budget of
around $150 billion, a workforce
of some 1.2 million employees,
and contracts with
more than 30,000 social enterprises.
It needed a set of
tools that would be both robust and flexible, and that could be used for decision making as
well as evaluation.

We started by scanning existing social value metrics, such as the
ones described in the table “10 Ways to Measure Social Value” on
page 41. We found hundreds of competing tools, of which foundations
and NGOs generally use one set, governments another, and
academics yet another. In addition to discovering this segmentation,
our survey suggested two more reasons why so few metrics guide
real decisions. First, most metrics assume that value is objective,
and therefore discoverable through analysis. Yet as most modern
economists now agree, value is not an objective fact. Instead, value
emerges from the interaction of supply and demand, and ultimately
reflects what people or organizations are willing to pay. Because so
few of the tools reflect this, they are inevitably misaligned with an
organization’s strategic and operational priorities.

The second reason that current measures of social value fail to
influence decision makers is that they conflate three very different roles: accounting to external stakeholders, managing internal operations,
and assessing societal impact. In the business sector, decision
makers use different tools for each of these tasks. An airplane
manufacturer, for instance, would use one set of metrics, mandated
by laws and regulations, to explain to external stakeholders how it
spends its money. The company would then use a second set of metrics
to allocate resources in the building of airplanes. (It is a brave
manager who would let investors see these internal accounts.) The
company would then use entirely different kinds of measures to
explain how its activities affect larger economic indicators such as
gross domestic product.

Yet in the social and public sectors, some proponents of new
social value measures claim that their metric can play all three
roles. Not surprisingly, and despite courageous efforts, these attempts
to do three things at once have resulted in the failure to
do any one of them well.

Here, I describe a better way to think about social value: the
product of the dynamic interaction between supply and demand in
the evolution of markets for social value. I then show how decision
makers in the nonprofit and public sectors can use these insights to
measure what can be measured without pretending to measure what can’t be. Finally I recommend better ways to make social value metrics.
My main advice is that nonprofits and foundations should resist
the current trend of developing assessment tools entirely separately
from public policy and academic social science, and instead should
collaborate across sectors.

Elusive Quarry

The failure of the social and public sectors to measure the value
they create does not stem from a paucity of intelligence or good
intention. Rather, it reflects four unavoidable complexities that
bedevil the measurement of social value. First among these is the
lack of hard-and-fast laws and regularities in the social field. Many
people would love the social field to be more like natural science,
so that they could definitely predict the effects of, say, a $10 million
investment in a crime prevention program.

But unlike molecules, which follow the rules of physics rather
obediently, human beings have minds of their own, and are subject
to many social, psychological, and environmental forces. Several
decades of involvement in evidence-based policymaking has
shown me that although evidence should inform all action, very
few domains allow precise predictions about what causes will lead
to what effects. The social sciences (including business) simply do
not have laws in the way that physics has. Even seemingly solid economic
principles, such as the rule that demand falls when prices rise,
have many exceptions.

A second reason that measuring social value is hard is that, in
many of the most important fields of social action—such as crime
prevention, childcare, and schooling—people do not agree about
what the desired outcome should be. In other words, the public
argues not only about social value, but also about social values. For
example, many people want to imprison criminals to punish them,
even when incarceration costs more and confers fewer benefits than
do alternatives to prison. Psychologists call this willingness to sacrifice
a lot to penalize others altruistic punishment.

Because people’s ethics, morals, and priorities vary, social value
assessments that look only at costs and benefits are bound not to
influence many members of the public and the politicians who
represent them. Philosophers (from John Dewey to Luc Boltanski)
have long recognized that societies are made up of competing and
conflicting systems of valuation and justification. But measurers of
social value have often tried to deny this.

Even without these problems, many social value metrics are inherently
unreliable. Measurements of social return on investment
(SROI), for example, often quite arbitrarily estimate costs and paybacks,
which dramatically affects the final calculated value. SROI
calculations can help in broad-stroke predictions, but they can’t
help with finer-grained decisions.

Revealed preference and stated preference methods are also notoriously
unreliable. Although they try to provide precise numbers,
they are not very rigorous about the means of deriving these numbers.
As a result, these methods confuse rigor with precision—a point that
REDF and others in the SROI field increasingly recognize.

A final reason that measuring social value is difficult is the problem
of time—estimating how much good an action will bring about many
years in the future, relative to how much it will cost to implement it now. In predictions of commercial returns on investment (ROI),
businesspeople use discount rates to account for the assumption that
a given amount of money will be worth less in the future than it is
in the present. With a 5 percent discount rate, for example, $100 of
today’s money will be worth only $35.85 in 30 years, and only $7.69 in
50 years. Many current measures of social value, such as SROI, likewise
use commercial discount rates—perhaps because of a mistaken
belief that treating social discount rates as equal to commercial ones
will make social value metrics seem more rigorous.

But it’s not clear why social organizations and governments
should use commercial discount rates, especially as these rates
radically devalue the future. Indeed, we should hope that the people
in these organizations give greater weight to the interests of
future generations than do commercial markets. A closer analysis
of discount rates suggests that they do.3 In health, many countries
apply a very low or zero discount rate, on the grounds that younger
generations should not be disadvantaged relative to older ones.
Governments ignore discount rates when investing in education
and defense technologies. And in climate change policy, a furious
debate has raged about what discount rates to apply—again in part
a moral argument about how to weigh the needs of future generations
against the needs of current ones. These examples reflect
my broader point: Social value is not an objective fact. Instead, it
emerges from the interaction of supply and demand, and therefore
may change across time, people, places, and situations.

Constructing Value

Borrowing practices from business and economics has led to many
mistakes in the measurement of social value. Yet these fields still
offer some important lessons for the field of social innovation.
For much of human history, philosophers and economists believed
that value was an objective fact. Aristotle thought that there was a “just price” for everything, for instance. And Karl Marx thought
that value came from labor.

But more recently, most economists have accepted that the only
meaningful concept of value is that it arises from the interaction of
demand and supply in markets. In other words, something is valuable
only if someone is willing to pay for it. This blunt approach upsets many
people because it implies that there may be no economic value in a beautiful
sunset, an endangered species, or a wonderful work of art. But this
definition of value is useful because it forces economists to observe real
behavior, rather than trying to uncover hidden realities.

The time is ripe for the social field to take an equally simple starting
point, and to view social value as arising from the interplay of
what I call effective demand and effective supply. Effective demand
means that someone is willing to pay for a service or an outcome.
That “someone” may be a public agency, a foundation, or individual
citizens. Effective supply means that the service or outcome works,
is affordable, and is implementable. I use the qualifier “effective” because
social problems will always invite simple supply and simple
demand. But to measure social value, the supplies and demands
must be effective in the senses described above.

Markets, conversations, and negotiations then link, on the one
hand, people and organizations with needs and resources, with, on
the other hand, people and organizations with solutions and services.
Social value metrics are useful if they give shape to these markets,
conversations, and negotiations.

In some fields, the links between supply and demand are mature.
For example, many voters are willing to pay taxes for police forces
and primary schools, and many governments are able to supply
these services. Likewise, many donors are willing to fund health care
for children in developing countries, and many local charities and
churches are able to deliver this care. In these domains, analyzing
social value is not difficult, because the links between what funders
want and what providers know they can offer is clear.

But for other social issues, the links between supply and demand
are missing. In some cases, effective demand may be lacking because
funders, politicians, or private citizens do not view a need as pressing
enough to warrant their resources. For example, some states are
unwilling to fund sex education or drug treatment. In other cases,
effective demand may be present—for instance, many governments
are willing to pay to reduce obesity—but the supply of cost-effective
interventions is limited. In these situations any descriptions of social
value are bound to be more tentative and exploratory.

In still other cases, both sides of the supply-demand equation may
be murky or complex. Many social policy makers, for instance, understand
that more holistic solutions often yield better results. But
holistic approaches necessarily have to deal with purchasers—that is,
demand—that are split across many different public agencies and NGOs,
each with its own view of what really counts as valuable. The supply
side may also be fragmented: Helping homeless people, for example,
may depend on the contribution of many different agencies to provide
therapy, alcohol treatment, job training, and housing. In these fields,
too, social value can become clearer only through iterative processes
that bring together supply and demand in deliberation and discussion.
Even the most brilliant researcher cannot measure or even describe
social value if she is not immersed in these discussions.

Healthy Numbers

All of these points have become particularly clear in our work for the
NHS, which is involved in everything from routine checkups, to surgeries,
to behavior-change interventions, to community programs.
The advantage of a single, integrated health service like the NHS is
that it has to be explicit about its demands, that is, what it needs and
what it is willing to pay for. The NHS’s effective supply side is also
reasonably easy to define, and it includes doctors, nurses, managers,
social enterprises, private providers, and members of the public.

To help the NHS make better decisions and allocate its resources
more effectively, we at the Young Foundation created a tool that
makes explicit the social value of various alternatives. Earlier on I
described the three very different roles metrics can play—external
accountability, internal decision making, and assessment of broader
social impact. The tool we developed focuses squarely on the second
of these goals. It attempts to capture the value that accrues to
the individual from being healthy, rather than sick; to caregivers; to the wider community (for example, from the control of infectious
diseases); and to the taxpayer.

The tool we created is not a simple computer program or calculator.
Instead, it is a framework for thinking about value. Like
many of the tools used to assess social value, this one requires a
series of judgments. The judgments fall into four main categories:
1) strategic fit (how well the proposed innovation meets the needs
of the health service); 2) potential health outcomes (including likely
impact on quality-adjusted life years and patient satisfaction); 3)
cost savings and economic effects; and 4) risks associated with
implementation.

When faced with a proposal, users of the tool apply a 0 to 5 scale
to rate the proposal on items in each of these categories. Proposals
range from a promising idea from a group of doctors or nurses, to
an idea that has already been piloted on a small scale or a venture
that is ready for scaling up. Users can also provide commentary
along with their ratings.

In some cases, decision makers can draw on strong data—for example,
evidence from a randomized controlled trial. In other cases,
they must rely on less certain numbers. To capture this variability,
the tool also includes measures of the reliability of the evidence on
which judgments are based. The visual presentation of the results
then makes judgments and their reliability very clear.

Once mastered, the NHS tool is quick to use and transparent. Multiple
users can interrogate the judgments, and in due course compare
them with what actually happened. It is also publicly available. Ten regional innovation funds (worth around $350 million in total) are
using it as a basis for decisions and encouraging applicants to use
the tool to assess themselves. Decision makers can also use the tool
to review each other’s work, to ensure consistency in their decisions,
and to communicate with other public agencies.

The net result of the NHS tool is a picture of social value that is
explicit about what’s valued and what isn’t; that doesn’t try to combine
everything into a single number; that is transparent and interrogatable;
and that is simple enough to help decision makers having to cope
with a large volume of examples. It also avoids the flaw of trying to
impose a single discount rate onto diverse measurements.

Making Markets

I describe this tool because it is one approach to operationalizing social
value that balances coherence, consistency, and simplicity with the
flexibility needed to cope with messy and complex phenomena. Developing
it was helped by the fact that the NHS is an organization with
clear supplies and demands. But for most NGOs, supply and demand
are fuzzier, and each field brings with it a different set of concerns.

For example, primary, secondary, and tertiary educational institutions
create value for students and the wider society. They rely on
a strong research base to decide which types of education deliver
which returns to whom. Vocational education, in contrast, presents a
different set of considerations. Certain kinds of skill may be of value
to only one sector, or to a small set of employers. A program offering
intensive support to a chaotic drug user will have a still more
complicated set of values, including value for the individual (both
financial and health-related), value for the community (for example,
from lower crime), as well as value for a wide range of public agencies
(from hospitals whose emergency services will be used less, to
police, prisons, and welfare agencies).

Seen through this lens, the job of funders is not to alight on one
particular method for measuring value. Although common frameworks
for thinking about value are useful, funders must adapt these
frameworks to the organization and field under consideration.

Indeed, the greatest contribution that funders can make is
often not to measure value, but to forge the links between supply
and demand that will later generate value. For example, they
can invest in effective supply by supporting promising projects
and collecting evidence of what works. They can invest in effective
demand by persuading governments to use their much
greater resources to pay for new services. And they can use their
convening power to connect purchasers and providers and then
encourage them to talk.

Foundations can also help less powerful players have a voice in
the market. Many groups, such as homeless people, migrant workers,
and people with mental illnesses, have clear needs but lack the
resources and political power to translate their needs into demand.
Foundations can turn this latent demand into effective demand. For
instance, several European foundations that support undocumented
migrants have developed the demand side of this emerging social
market by encouraging larger NGOs and public authorities to allocate
resources (for example, for housing and health care) to it. On
the supply side, these foundations have funded promising projects
that are more effective at meeting the needs of this group.

Some foundations are likewise developing the market for addressing
elder abuse. On the demand side, they have funded research on
the extent of the problem and influenced commissioners to allocate
resources and attention to it. On the supply side, they have supported
innovative programs to prevent or mitigate abuse.

In both cases, governments’ resources vastly outweigh those of
foundations and NGOs. This is almost always the case. Just about
anyone wanting to make a big social impact has to engage with the
worlds of politics and public provision.

Good Accounts

The field of social innovation can learn some lessons from business
and economics. But it should not be naive. As the collapses of Enron
and Lehman Brothers revealed, even such seemingly objective
metrics as profit are not the facts they appear to be in economics
textbooks. And in business, accounts are just that: accounts. They
are ways of explaining what is being done, with an eye toward the
often conflicting interests of investors, managers, regulators, and
consumers. They involve judgments as well as facts.

Anyone who wants to finance social goods and anyone who
wants to provide them should use metrics to clarify how inputs can
contribute to outcomes, as well as to clarify choices and trade-offs.
But they should abandon metrics that obscure these choices or that
pretend to offer a spurious objectivity. And they should use metrics
only in proportionate ways. It’s not sensible for a small NGO to
invest scarce resources in apparently elaborate estimates of social
value—not least because these estimates are bound to crumble under
serious scrutiny.

Meanwhile, larger NGOs that do need measures of social value
should clearly distinguish between those that are primarily about
external accountability, those that help internal management, and
those that support assessments of broader patterns of social impact.
If an organization is using the same method for all three, its findings
are almost certainly flawed.

People involved in funding social value, whether at the stage of
promising innovations or of large-scale practice, likewise need sharper
common frameworks. Greater use of these shared frameworks would
be more valuable than proliferation of ever more assessment tools.
Building on these frameworks, what matters is the quality of the discussion
and negotiation, and the depth of the learning several years
later, when participants reflect on what worked and what didn’t.

Notes

1 This article draws on several main sources, including Geoff Mulgan, Gavin Kelly, and
Stephen Muers, Creating Public Value, London: U.K. Cabinet Office, 2002; and Geoff
Mulgan, Gareth Potts, Matthew Carmona, Claudio de Magalhaes, and Louie Sieh, "A
Report on Value for the Commission on Architecture and the Built Environment," London: Young Foundation, 2006. The Young Foundation Web site also provides details
about many measurement methods.2 The following books provide a good overview: C.J. Barrow, Social Impact Assessment:
An Introduction, London: Hodder Arnold, 2000; Henk A. Becker and Frank Vanclay
(eds.), The International Handbook of Social Impact Assessment, Cheltenham, U.K.:
Edward Elgar Publishing, 2006; "Identifying the Environmental Causes of Disease:
How Should We Decide What to Believe and When to Take Action?" Academy of
Medical Science, 2008; John Dewey, Theory of Valuation, University of Chicago Press,
1939; and Luc Boltanski, Laurent Thevenot, and Catherine Porter, On Justification:
Economies of Worth, Princeton, N.J.: Princeton University Press, 2006.3 For a fuller analysis of discount rates, exponential rates, and hyperbolic rates, see the
chapter on value in my book The Art of Public Strategy.

Geoff Mulgan is director of the Young Foundation; a visiting professor at the
London School of Economics, University College London, and the University of
Melbourne; and chair of the Carnegie Inquiry into the Future of Civil Society in
the United Kingdom and Ireland. He was previously director of the U.K. government
strategy unit, and head of policy for Prime Minister Tony Blair. His most recent
book is The Art of Public Strategy, published by Oxford University Press (2009).