This special supplement includes six articles that address basic principles and practices that inform efforts to monitor performance, track progress, and assess the impact of foundation strategies, initiatives, and grants. The supplement was sponsored by the Aspen Institute Program on Philanthropy and Social Innovation and underwritten with a grant from the Ford Foundation.

Across every sector of society, decision makers are struggling with the complexity and velocity of change in an increasingly interdependent world. The context of decision-making has evolved, and in many cases has been altered in revolutionary ways. In the decade ahead our lives will be more intensely shaped by transformative forces, including economic, environmental, geopolitical, societal, and technological seismic shifts.1

As part of its response to this global dynamism,
the Rockefeller Foundation has
translated its 1913 mission of promoting the
well-being of humanity into two overarching
goals: expanding opportunity through
more equitable growth, and strengthening
resilience to acute crises and chronic
stresses, whether man-made or ecological.
Our vision is a world in which globalization’s
benefits are more widely shared and
the inevitable challenges that accompany
a world that is fast changing, diverse, and
complex are more easily weathered.

The Rockefeller Foundation structures
its work around time-bound cross-sectoral
initiatives that seek innovative solutions
and support enabling environments to
bring about change. The foundation’s structure
reflects its view that today’s problems
and solutions are multi-dimensional in
scope and nature, and that they require
multi-disciplinary responses at the intersection
of fields.

Just as the Rockefeller Foundation’s
approach to philanthropy has evolved, so
too has its approach to evaluation. With its
mission to improve the well-being of humankind,
its focus on impact, and much of
its grantmaking in developing countries,
the Rockefeller Foundation is committed
to evaluation practices that are rigorous,
innovative, inclusive of stakeholders’
voices, and appropriate to the contexts in
which the foundation works. This article
discusses how the Rockefeller Foundation
integrates the views of developing-region
evaluators into its evaluation approaches,
and highlights five key strategies:

Engaging stakeholders to develop
shared outcomes.

Expanding capacity through use of
non-staff monitoring and evaluation
specialists to partner with grantees.

Sharing knowledge through learning
forums and communities of practice.

Strengthening developing country
evaluation practice and ownership of
results.

Developing innovative methods and
approaches to evaluation and learning.

Rethinking, Reshaping,
and Reforming Evaluation

In November 2011, the Rockefeller Foundation
brought together leaders from philanthropy
and development at the Future of
Philanthropy and Development forum, held
at its conference center in Bellagio, Italy.
In one of the keynote papers, Evaluating
Development Philanthropy in a Changing
World, Robert Picciotto, former vice president
at the World Bank and now professor
at King’s College London, squarely tackled
the role of evaluation in philanthropy and
development. “The changing context and
thinking on development has profound
implications for development evaluation
itself, and for the contribution evaluation
can bring to the empowerment of people;
and the effectiveness of development interventions
by national governments and
international partners and, increasingly, by
non-state actors—foundations, philanthropists,
and agencies that promote investing
for impact.”

The value of evaluation must ultimately be judged by its usefulness in helping to improve outcomes for target beneficiaries.

Picciotto continued, “Pressing human
needs are not being met by an official aid
system that is short of resources, catering
to multiple interests, and hobbled by massive
coordination problems. By contrast,
private giving for development is growing
and has proven nimbler and more results-oriented
than official aid. However, the
philanthropic enterprise will not fulfill its
potential unless it identifies and taps into
its distinctive comparative advantage and
coordinates its interventions with other development
actors; embeds evaluation in its
processes to achieve operational relevance,
effectiveness, and efficiency; and demonstrates
that it is accountable and responsive
to its diverse stakeholders.”

Developing country evaluation leaders
have also articulated the need for a new approach
to evaluation and the role it plays in
improving the wellbeing of humankind—in
particular, the lives of the poor and vulnerable
in developing countries. At the January
2012 gathering of the Africa Evaluation
Association’s biannual conference in
Accra, Ghana, African evaluation leaders
and policy makers highlighted five steps
foundations and development agencies
must take if they aspire to play a meaningful
role in social transformation.

Broaden the inclusion of key stakeholders
in evaluation. Only when the
voices of those whose lives we seek to
improve are heard, respected, and internalized,
will we be able to effectively
evaluate what success should look like
for the people we are most concerned
about. Foundations and agencies need
to take practical steps to include key
stakeholders in the design of, conduct
of, and learning from evaluation.

Regard evaluative knowledge as
a public good and share it widely.
Learning with our partners and
stakeholders about what works and
what doesn’t work should be seen
as a global public good not limited
to boards and program teams, but
shared widely with grantees, partners,
and peers.

Address evaluation asymmetries
between developed and developing
regions. The majority of human and
financial resources for evaluation emanate
from agencies and foundations
based in the developed world. With
evaluators from developing countries
playing a minor role, if any, many of
them do not get sufficient experience
to move into leadership roles. Mentoring,
coaching, and training can
strengthen the role, capacity, and resources
of developing-country evaluators
so that they can play key roles
in conducting and using evaluation
results for social transformation and
accountability in their own countries.

Broaden the objects of evaluation to
learn more beyond the individual
grant or project to a more strategic assessment
of portfolios of investments,
policy change, new financing mechanisms,
and sector-wide approaches
that tell us more about what works and
what does not in different contexts.
Framing evaluation to take into account the drivers of unsustainability and causes of the challenge being addressed provides greater learning than narrower evaluations that focus only on the funder’s specific intervention.

Invest in the development and application
of innovative new methods and
tools for evaluation and monitoring that
reflect multidisciplinary and systems
approaches to problems and complexity;
invest in methods that assess network
effectiveness and policy change;
and use and adapt new technology to
enable stakeholders to provide close to
real-time data and feedback.

The Rockefeller
Foundation’s Approach

With its long history of supporting developing
country institutions, the Rockefeller
Foundation has responded to the call to action
from developing-region evaluators by
adopting the following approaches to planning,
monitoring, and evaluating its work.

Shared Outcomes | An important underpinning
of the Rockefeller Foundation’s
initiative-based approach is a fundamental
recognition that the world’s greatest challenges
can’t be solved alone. These challenges
involve a complex mix of actors that
are often globally interdependent across sectors
and geographies. Networks, alliances,
and coalitions of diverse stakeholders from
governments, foundations, civil society, and
business are increasingly seen by the foundation
as a more powerful way to mobilize
the vast range of resources and actions required
to bring about sustained and transformational
change on a significant scale.

Increasingly, the Rockefeller Foundation
brings together grantees and partners
from developed and developing countries
to establish a common vision of the problem,
outcomes, and indicators for success.
Grantee agreements now include reference
to the common vision of results and shared
outcomes to which the grantee contributes,
and foundation teams are expected to manage
portfolios of grants and relationships
with grantees towards that common vision.
This shared-outcomes approach forms the
basis for ongoing monitoring, evaluation,
and reporting, and for learning dialogues
with grantees and partners. (See “Shared
Results Framework.”)

Monitoring and Evaluation | Most
foundations have capacity limitations on
the amount of time that can be devoted to
monitoring and learning with grantees and
partners, visiting field projects, and working
collaboratively—activities that we know contribute
to greater collaborative learning and
effective relationships. Recognizing these
limitations, the foundation awards grants
to monitoring and evaluation (M&E) groups
and specialists in developing and developed
countries who act as monitoring partners, or
what we call “critical friends,” 2 throughout
the life of initiatives (typically, a five- to six-year
period). They work with grantees to
identify key learning questions, help to set
up monitoring systems, and provide support
in analyzing monitoring data. The most significant
feature of the critical friends is that
they build trust with grantees and partners
to ask tough evaluative questions, and they
support grantees in seeking and using feedback
to make improvements throughout the
life of the initiative. Periodic evaluations are
conducted by independent teams to provide
an objective assessment of progress toward
outcomes and impact.

For example, the India-based nonprofit
Participatory Research in Asia, in collaboration
with the Ghana-based Institute for Policy
Alternatives, works alongside Shack/Slum
Dwellers International (SDI) which directly
represents millions of urban poor slum dwellers
in 33 countries. The aim of this critical
friend partnership is to strengthen the participatory
learning, monitoring, and evaluation
systems and abilities of the urban poor
networks to better capture and systematize
learning and strengthen accountability with
the goal of empowering the urban poor to
achieve wider positive impacts. The critical
friend role underpins a belief that federations
of the urban poor are capable of changing
their own situation for the better. As a result
of this partnership, SDI has strengthened its
ability to democratize learning, monitoring,
and evaluation—continuing to place the tools,
responsibility, and ability for change in the
hands of its members.

This figure illustrates the framework around which Rockefeller Foundation staff,
grantees, and partners develop a common vision of the results and impact that they
seek to achieve collectively. The top frame represents the mission and strategy of
the foundation—promoting the well-being of humanity in two overarching goals: expanding
opportunity through more equitable growth, and strengthening resilience.
The middle frame represents the medium-term outcomes that the foundation seeks
to achieve during the life of the initiative (these change from initiative to initiative).
The lower frame represents the work that grantees, partners, and staff do in their individual
organizations to collectively bring about outcomes and ultimately improve the
lives of beneficiaries. These shared results frameworks anchor the ongoing dialogue
with grantees about progress toward achieving this vision and their contribution to
the shared outcomes. It also serves as a framework for managing portfolios of grants
and monitoring changes during the life of the initiative.

Learning Forums and Communities of
Practice | Most of the Rockefeller Foundation’s
initiative teams convene grantees and
partners annually to review progress, highlight
lessons and challenges, celebrate successes,
and identify improvements needed.
Through these forums grantees learn from
others in the field, meet new resource people,
and adjust their strategies going forward.
Although this practice does not guarantee
impact, it increases the likelihood of it by
creating a greater sense of ownership and
shared outcomes, and it increases leverage
by connecting grantees with new resource
people, funders, and mentors. Increasingly,
M&E grantees produce high-quality knowledge
products as a public evaluation good
to highlight what works, what does not, for
whom, and under what conditions. Our aim
is to establish with grantees a body of collaborative
knowledge, shared lessons, and a
culture that values evaluation as a resource
for learning as well as for accountability.

For example, the Rockefeller Foundation
has aligned with the South East Asia Community
of Practice in Evaluating Climate
Change Resilience (SEA Change), facilitated
by the nonprofit organization Pact, to
work on urban climate change resilience in
ten Asian cities. This community of practice
brings together evaluators, program
managers, grantees, and policy makers
concerned with learning what works in interventions
aimed at adapting and building
resilience to the effects of climate change
and extreme weather events in Southeast
Asia. Resources and lessons are shared
through online learning, onsite convening
of SEA Change participants, and coaching,
mentoring, and training provided by members
of the community of practice throughout the countries of Southeast Asia.

Addressing Asymmetries | The Rockefeller
Foundation is supporting the formation
and strengthening of regional developing-country networks and the first-ever
regional institutions to train, coach, and
mentor evaluators, and to partner with evaluators
from other regions. Through these
platforms and networks, the foundation
aims to help rebalance the asymmetries
of choice and opportunity for developing-country
evaluators to control the evaluation
process in their own localities and to
improve the quality of evaluation by partnering
with evaluation leaders globally.

One example of this is the African Evaluation
Association (AfrEA), a pan-Africa umbrella
organization comprising more than 25
national M&E associations in Africa, and a
resource for individuals in countries where
national evaluation bodies do not exist.
AfrEA, which has more than 1,000 evaluators
from all regions of Africa, receives
Rockefeller Foundation funding to enable
the formalization of its organizational, operational,
and management structure, and
to build communities of practice among
its membership to tackle the most pressing
evaluation challenges on the continent.

The Centers for Learning on Evaluation
and Results, located in Africa and Asia,
are another example of an effort aimed
at addressing asymmetries in evaluation
in developing countries. Together with
a consortium of funders committed to
building developing country capacity for
taking charge of the evaluation agenda in
their regions, the foundation is supporting
regional centers3 in East and West Africa
and South Asia to strengthen their skills,
networks, and experience in monitoring
and evaluation and results-based management
capacity of public, private, and civil
society development in the global south.

New Methods and Approaches | Traditional
evaluation methods and approaches
to learning, accountability, and feedback
have not kept pace with the advances in
technology and social media. The majority
of evaluation practice is still largely paper-based
despite great strides in technology,
interactive web-based platforms, and multimedia
tools that make real-time feedback
from grantees and beneficiaries possible
and accessible. The Rockefeller Foundation
and its partners learned a great deal from
Ushahidi, an open-source crowdsourcing
project that allows users to send crisis information
via mobile devices to map reports
of violence or suffering. Inspired by the potential
of these kinds of tools to democratize
evaluation information, increase transparency,
and lower the barriers for individuals
to share information and stories, the foundation
is supporting a number of innovative
approaches to evaluation.

We are privileged to work in an expanding field in which our evaluation findings can change lives for the better.

One example is GlobalGiving’s Story
Telling project, an innovative way to gather
local feedback from people in developing
countries and to share it with communities,
implementing organizations, and donors to
create real-time feedback. With support from
the Rockefeller Foundation, GlobalGiving
successfully deployed a network of people
in Kenya, Uganda, and Tanzania that has
generated more than 20,000 tagged narratives
from thousands of people. Some of
GlobalGiving’s partners are deriving actionable
intelligence from these stories, and
GlobalGiving is discovering patterns in the
stories that inform its own operational and
strategic decision-making processes.

Another example is BetterEvaluation, an
online interactive community of evaluation
practice developed by the Royal Melbourne
Institute of Technology in partnership with
the Institutional Learning and Change Initiative
and the Overseas Development Institute,
with support from the Rockefeller
Foundation and Pact. BetterEvaluation
provides advice, online support, and good
practice examples to evaluators in developing
and developed countries.

Reshaping Development
Evaluation

Philanthropists and development practitioners
have a golden opportunity to join
together with grantees and partners in developing
countries to reshape evaluation
to better respond to global change and to
serve our missions and goals more effectively.
To do this, we must be prepared to rethink
and reshape our evaluation practice
in at least four ways. We must:

Embrace a broader set of voices in
framing our approaches to evaluation.

View collaboration and partnerships
between developed and developing
areas as mutually beneficial toward a
common goal of expanding and sharing
evaluation knowledge as a public
good aimed at achieving better development
outcomes.

Recognize the need to address issues
of accountability, transparency, ethics,
culture, and independence.

Address asymmetries in individual
and institutional capacities for undertaking,
driving, and owning evaluation
in developing regions by promoting
opportunities for professional excellence,
networks, and sustained global
partnerships in the discipline of development
evaluation.

The value of evaluation must ultimately
be judged by its usefulness in helping to
improve outcomes for target beneficiaries.
The quest for impact is currently in the
spotlight among foundations and development
agencies as we seek collectively
to maximize the positive benefits of our
resources. We are privileged to work in
an expanding field in which our evaluation
findings can change lives for the
better. Together with our peers, partners,
and grantees, we can and should rethink,
reshape, and reform the practice of evaluation
to better meet that challenge.

Notes

1 For more discussion of this topic, see Global Risks2012, Seventh Edition, World Economic Forum, 2012.2 The term “critical friend” refers to a partner that
builds trust and engenders a reflective evaluative
culture that is able to provide both negative and
positive feedback to the grantee in a supportive
way throughout the life of the work.3 In South Asia The Jameel Poverty Action Lab
South Asia at the Institute for Financial Management
and Research, India, with the Center for
Economic Research in Pakistan; In Africa the University
of Witwatersrand with the Kenya Institute
of Administration and Ghana Institute of Management
and Public Administration.

Judith Rodin is president of the Rockefeller Foundation.
Before joining the foundation, she was president of
the University of Pennsylvania, and preceding that, provost
of Yale University. Rodin is on the board of Citigroup,
Comcast, and AMR.

Nancy MacPherson is managing director, evaluation,
at the Rockefeller Foundation. Before joining the
foundation she spent 25 years in development evaluation
for international nonprofit organizations in Asia and
Africa and for the United Nations.