Scoping Study: What Works in Protection and How Do We Know?

Executive Summary

Ever since accountability reforms were folded into the aid sector in
the 1990s, humanitarian organizations and the larger relief system have
developed and improved their ability to evaluate the impact of their
work. Relief organizations have, in general, found it easier to measure
the impact of their interventions in relation to material needs than
activities geared to enhancing protection. For this reason, this scoping
study asks “what works in protection and how do we know?” Three related
questions will be discussed in the following order. The first is how to
define humanitarian and human rights protection. The second is how to
define success for different types of humanitarian and human rights
protection interventions. The third question is how to measure the
impact of different protection-oriented interventions.

Scope and Methods of the Scoping Study

The primary responsibility for protecting civilian populations from
violence and other forms of harm rests with national state authorities
(e.g., the police, armed forces or the judiciary), international state
actors (e.g. the UN, regional organizations, international justice
institutions) and non-state authorities that control territory. A key
objective for humanitarian and human rights actors is to change the
policies and behavior of such primary “duty bearers” who have or may
have a negative effect on civilian safety and well being. But the scope
of humanitarian protection organizations is significantly broader. They
implement specific measures to mitigate protection risks to communities
and affected individuals and deliver specialized services to address the
after-effects of violence and other patterns of harm. The research
focus of this scoping study is limited to defining and reviewing the
effects of different protection activities implemented by humanitarian
and human rights organizations. However, while it is not the ambition
here to assess the effectiveness of non-humanitarian protection efforts,
this report does discuss the potential influence of political and
military actors on the larger success or failure of humanitarian and
human rights protection.

This scoping study was commissioned by DFID as a desk study and
undertaken by a team of two GPPi researchers and one independent
consultant. Given that there is great variation in the way ‘protection’
is defined or understood at the operational or program level, the team
reviewed 173 documents, including academic works, evaluation reports and
other grey literature, such as “how to” handbooks and guidelines,
protection standards or policies. This includes 12 articles discussing
the effectiveness of interventions in related fields, such as
peacekeeping, peacebuilding or child protection in developments settings
(see chapter 2). The study also draws on 40 semi-structured interviews
with key informants from the UN, the Red Cross/Crescent Movement,
non-governmental organizations, academia and Western donor governments.

Summary of Protection Trends

In the context of humanitarian action, protection was traditionally
the preserve of international law specialists and a few organizations
mandated by international treaties and United Nations (UN) resolutions.
After the end of the Cold War a diverse range of humanitarian and human
rights organizations started to deploy staff to emergency settings where
they engaged in efforts to enhance the protection of civilians. Today,
protection has become an important element of the mission statement of a
large number of humanitarian actors. Furthermore, the contextual scope
of protection activities has broadened: In the past, only situations of
armed conflict were seen as creating protection challenges, whereas now
there is an increased recognition of protection needs in disaster
settings associated with natural hazard events. Many of the latter also
occur in situations of weak or contested governance or armed conflict
settings.

However, although there is more attention to protection than before,
this scoping study revealed a tendency to consolidation in the
protection sector. After more than two decades of continuous expansion
of protection activities and multiplication of actors, “the pendulum may
be swinging back,” as one interviewee put it. Incorporating protection
perspectives into the design and delivery of relief programs is regarded
as a minimum obligation by most humanitarian organizations but there is
also a growing recognition that only a limited number of actors have
the experience and will to engage primary duty bearers (i.e. state
forces and armed groups) in a protection dialogue.

Defining success for different types of protection interventions

In this study, the term “humanitarian and human rights protection” in
emergency settings refers to a set of activities that are concerned
with countering violence and other patterns of harm such as sexual
exploitation, discrimination, forced displacement and separation of
families. Different protection activities are classified into three
distinct types of interventions:

This involves delivering material and non-material remedy to victims
of violence and other patterns of harm and helping them to gain access
to reparation and specialized care (e.g. medical assistance for rape
victims, psychosocial counseling, legal aid, family tracing services).
Such remedial action is essentially responsive. It attempts to mitigate
suffering in emergency situations and does not attempt to reduce the
incidence of particular patterns of violence or harm. To measure the
effectiveness of such interventions, it is necessary to determine
whether they helped to restore the dignity and well being of assisted
victims and to prevent further harm.

Protection Intervention Type 2: Reducing Risk Exposure

This involves implementing specific risk mitigating measures to avoid
or reduce the immediate exposure of civilian populations to violence
and other patterns of harm. Mine-risk awareness campaigns are an
example. Protection mainstreaming also falls under this category.
Closely related to the “do no harm” principle, protection mainstreaming
refers to efforts aimed at incorporating protection concerns into
overall relief programming. While preventive in nature, risk-mitigating
measures do not seek to address the deeper causes of violence and harm
against civilians rooted in the policies or behavior of relevant state
authorities or armed groups. Type 2 protection interventions prove
successful when they help to reduce the incidence of particular
incidents of harm (e.g. sexual violence) or physical injury (e.g. caused
by anti-personnel mines) in specific geographic locations.

Effective protection goes beyond efforts to reduce risk exposure and
remedy the after-effects of sustained harm. It aims to secure an end to
ongoing patterns of violence and harm that are detrimental to civilians
and to inhibit their future occurrence. To persuade actors engaged in
harmful practices to change their policies and behavior, humanitarian
and human rights organizations carry out different forms of advocacy.
The extent to which private and public advocacy helps to reduce the
incidence of patterns of violence and harm in the affected country or
sub-national region is the main determinant of success.

The scale of ambition that underlies each intervention varies
considerably. For each of the three protection interventions, success is
also defined differently. Humanitarian protection initiatives may be
highly effective, for instance, in providing specialized care to the
victims of violence – and thereby contribute to reduced civilian
suffering – but they may fail to eliminate or at least reduce certain
risks associated with harmful behavior. Thus, unqualified claims of
general “failure to protect” need to be critically examined. Any attempt
to determine the relative success or failure of humanitarian and human
rights protection needs to distinguish between the three different types
of interventions.

What Works in Protection: Key Findings Emerging from the Literature Review

The literature review revealed only a few sophisticated attempts at
measuring the success of different types of protection interventions.
Three main reasons account for the scant evidence on what works in
protection across different contexts:

Quantity of information: Reviewed works focus on implementation
challenges linked to capacity gaps, coordination issues and other
practical matters. Questions of impact are addressed at the margins.

Quality:
About half of the different academic works reviewed lack an explicit
research design and method, but clarity on design and method is a
precondition for generating reliable data.

Comparability: They
lack a common conceptual framework to assess success in protection
restricts the comparability of the findings that are presented in
evaluative reports and scholarly enquiries.

It is generally easier to find negative examples of humanitarian
protection efforts than positive ones – a central finding of the review.
Analyzing past mistakes may yield important insights on what might
work, but it does not provide concrete evidence of the circumstances
under which certain types of interventions do prove effective.

Questions for Further Research

The report concludes with research recommendations based on the
review of the literature and consultations with interviewees. The three
research questions outlined below have been identified to address the
current lack of a common methodology for indicator development and, more
generally, to further our understanding about what works in protection:

What are common protection problems and related modes of action used across different contexts and organizations?

What
are appropriate methods and processes for determining impact and change
triggered by different types of protection interventions?

What are common external factors that enhance or limit the success of different protection interventions across contexts?

Some of the challenges in measuring and attributing success in
relation to humanitarian and human rights protection are similar to
those encountered in complex development interventions. Where relevant,
the final chapter refers to the wider literature pertinent to this topic
and discusses the potential value and relevance of solutions devised
for development interventions and related fields, such as international
peacebuilding.