Third Party Monitoring in Volatile Environments - Do the Benefits Outweigh the Risks?

Increasingly... third party monitoring is being used more as a risk management tool in countries considered too volatile for development agencies to deploy their own staff. This raises important quality, ethical, and technical issues that deserve a wider debate.

The use of third party monitoring in development projects has gained increasing prominence in recent years, driven by demand for more robust accountability mechanisms. Typically, the goal is to bring an independent perspective on project performance by using outside parties who are neither direct beneficiaries or part of the project’s management structures.

Increasingly, however, third party monitoring is being used more as a risk management tool in countries considered too volatile for development agencies to deploy their own staff. This raises important quality, ethical, and technical issues that deserve a wider debate.

First, it is useful to understand the broader context driving this trend. Around 20 percent of the world’s poor live in fragility and conflict-affected situations and the number is expected to double by 2030. As a result, global attention and development aid is increasingly being focused on supporting prevention and early response interventions to address the drivers of conflict. To this end, the International Development Association – the part of the World Bank that helps the world’s poorest countries – has committed to doubling its financing for fragile and conflict affected states. At the same time, restrictions and security concerns around deploying its own staff in fragile countries, have pushed the World Bank to increasingly rely on various forms of third party monitoring techniques to collect data and to report project results.

Third Party Monitoring fills a critical evidence gathering gap. Without some form of project verification, development partners, including the Bank Group would not be able to justify or account for financial support to conflict affected areas. Third party monitoring has been an integral feature of the Bank’s Afghanistan portfolio, for example, enabling real time data collection and course correction. Local and expat subject matter experts, contracted by executing agencies, have promoted accountability through thousands of site visits and local data gathering. Geared mainly towards verifying infrastructure quality, this tool has helped to pinpoint problem areas and validate field results. Similar techniques have been deployed in Iraq, Yemen and Mali as well.

But third party monitoring carries risks and limitations. Recent research conducted by the Secure Access in Volatile Environments (SAVE) program, funded by UKAID, suggests that third party monitoring works best when it is used as a last resort, in conjunction with agencies’ internal monitoring systems, and when it is limited to exceptional areas with constrained access. Third Party Monitoring is mainly used to gather output data: it has limited ability to measure outcomes or impacts or to determine why or how a project can achieve better results. As pointed out by Gaarder and Annan (2013), it is both possible and important to carry out impact evaluations even in conflict-affected settings. The Bank has commissioned influential evaluations in conflict affected areas, like the Randomized Impact Evaluation of Afghanistan's National Solidarity Program. It has also innovated with the use of high-frequency surveys in Somalia to establish statistical baselines and to monitor cost-effectiveness. Third Party Monitoring should supplement, rather than come at the expense of, these deeper more intensive data analysis methods that have been implemented by the Bank in fragile and conflict affected areas.

It should also not come at the expense of learning whether aid may be contributing to, or exacerbating, conflict. Unfortunately, little is known about how we can deliver aid to conflict-affected areas without exacerbating conflict. A recent systematic review of the causal impact of development aid on violence in countries affected by civil war found that aid in conflict zones is more likely to exacerbate violence rather than to dampen it. Two recent papers summarizing the aid experience in Afghanistan and Iraq, respectively, found small program size to be an important factor for success. In the Afghanistan paper, small-scale programs were found to be relatively more successful than large-scale ones because the latter tended to create unrealistic expectations and were more subject to corruption and targeting by insurgents, undermining their near-term effectiveness. Recent evidence from Iraq suggests that smaller projects that are coordinated with security forces tend to be more successful (Berman et al.). More research on this important topic will hopefully allow donors to design aid interventions in conflict-affected areas that effectively improve livelihoods without exacerbating conflict.

Ethical questions remain unresolved. One area that needs close attention is what ethical protocols donors apply to third party monitoring. Should third party monitors be asked to conceal their identity in an effort to promote their safety? What is the responsibility of the donor in the event that a third party is identified or, worse, detained or killed? Eighty percent of the estimated 208 aid workers killed, kidnapped or seriously wounded worldwide in 2016 were local – a figure that points to the risks involved. Other questions pertain to equity. In severely conflict affected settings, what is fair pay? And, for the sake of data accuracy, what protocols need to be put in place to mitigate the risks associated with conflict of interest, or collusion, between the monitor and the community?

As the donor community ratchets up its investment and attention towards assisting the world’s most vulnerable populations living in the most insecure places, it will need to widely debate the means by which it supports information flow both in and out of these areas. The need to know the effect that aid is having on the ground in these environments is imminent. Lives are depending on it.

About the image:
Irrigation Restoration and Development Project. An adjustable gate which allows water distribution and division in the canal. Intifat Village,shakardara, Kabul. Afghanistan. 13/1/2015. Photo: Ghullam Abbas Farzami / World Bank

1) It is an interesting article covering various factors of third party monitoring tool .
2) Third party monitoring is assumed to have capability to expose "Elite Capture" in general by Top Management people , Bureaucrats and Politicians etc .
3) In the CASE of SIPP-II and NJLIP project it is worth watching that how "Elite Capture " could fool around Third Party Monitoring and it started helping manipulation and corruption by "Elite Capture" instead of helping the purpose of the project.
4) Third party monitoring tool need to be more strict and a kind of enforcing to eradicate poverty in corrupt countries like Bangladesh.

I will give my usual comment. While it is good to have these debates, and yes, donors and implementing agencies must do better in all aspects of their missions, what is missing again is the involvement of the world's poor. Now while there is a difference between post-conflict and post-disaster interventions and longer-term development projects, it is my view that poor people are quite capable of making informed decisions about what are their problems and priorities, as well as solutions. If more trust is given to them, the results speak for themselves - and you might just save on the considerable overheads of the pure expert-led top down approach. http://anorthumbrianabroad.blogspot.com/2015/03/smarter-aid-not-more-ai…

Thank you for this insight! The destructive impacts of international development aid on poor communities have been widely and deeply exposed within academic circles since the eighties. It's surprising yet assuring to find it coming from the World Bank Group though!

Very interesting article. It deals with monitoring (understood as a critical and independent process accompanying project implementation and that feeds-back to the project implementing authority and the project financier) but automatically prompts the question about how (and by whom and according to which standards) project due diligence and appraisal have been carried out? And, would such thinking equally apply to evaluation as well?

Secondly, the article realistically confines the reduced monitoring scope to output (i.e. "limited ability to measure outcomes or impacts). One wonders whether the non monitor-ability of intended higher level achievements (in a sense, an intervention's raison d'etre) should become a critical due diligence concern that, eventually, might/should lead to declining the intervention in the first place - as harsh as it is? This also, since the intervention could otherwise well feed the opposite, un-wanted "dis-objectives".

In other words, if the "relevance" criteria [according to the OECD-DAC evaluation criteria] can't be determined to a satisfying degree - neither during due diligence, implementation, completion and post-completion - the intervention idea from a development aid perspective should be abandoned.

Very useful post. Monitoring is v development in one country I am familiar with--Mali--even in the regions that are not hit by the current political crisis (central and northern). Projects that focus on public infrastructure and basic services in rural Mali are playing a very important role in building and maintaining political legitimacy and citizen involvement in decentralized local governance. While decentralization in Mali has had much success, it remains hampered by a lack of transparency and control of infrastructure funds and management. If third party monitoring were to take a larger role in Mali due to ongoing risk and instability, it should be done carefully and in such a way that it supports rather than undermines local institution building.

Connect With IEG

The Independent Evaluation Group evaluates the work of the World Bank Group to find what works, what doesn't, and why. IEG evaluations provide an objective assessment of World Bank Group results, and identify lessons learned from experience. Through independent evaluation, IEG is helping the World Bank Group achieve its twin goals of eradicating extreme poverty and boosting shared prosperity.