Menu

Recommendations to improve evaluations of research capacity strengthening

TDR news item 7 May 2014

Two new studies on the monitoring and evaluation of health research capacity strengthening (RCS) initiatives have been published. The studies, which appear in the April issues of Health Research Policy and Systems and Health Policy and Planning, draw on work done by the TDR-based ESSENCE on Health Research initiative, a collaborative framework between funding agencies to scale up research capacity.

The authors, among them TDR’s manager of partnerships and governance Dr Garry Aslanyan, identified four ‘key tensions,’ including the extent to which funding recipients should participate in evaluating their own health RCS efforts; whether the focus of evaluations should be on understanding processes or on measuring impact; whether the purpose of evaluations should be to demonstrate accountability or to enhance knowledge; and whether short-term funding to conduct and evaluate projects can be reconciled with the need for long-term sustainability.

RCS initiatives have proven difficult to evaluate because of their complexity, their long timescales and the diversity of contexts in which they’re implemented, resulting in a lack of robust evidence on what works in conducting them. In 2011, ESSENCE members sought to address this evidence gap by collaborating on the production of a planning, monitoring and evaluation framework aimed at harmonizing good practices. The authors examined that PM&E framework as well as more than 50 reports on evaluations conducted by funding agencies as part of an effort to not only identify the tensions but also identify ways of avoiding them.

Steps that can improve evaluation effectiveness

The authors suggest steps that can be taken to improve the effectiveness of health RCS evaluations. For example, they write, by sharing and rapidly assimilating knowledge alongside, not after the evaluations, funding agencies can reduce the typical six-year time lag between end-of-project evaluations and the incorporation of lessons learned. To promote sustainability, funders should focus health RCS around specific research questions, engage in strategic partnerships, and choose partners with a track record of attracting funding.

Another critical recommendation, the authors report, is the need to actively engage all stakeholders, including both funders and funding recipients, in all stages of the evaluation process. This inclusive approach can facilitate the face-to-face dialogue found to be particularly useful. Similarly, they write, benchmarking and sharing of experiences through networking and exchange visits were reported to enhance informal learning, while regular participatory evaluation or self-assessment and rapid assimilation of findings could contribute to lesson learning for continuous improvement.

Research has shown that strengthening research capacity in low- and middle-income countries is one of the most effective ways of advancing their health and development. In recent years, funders have made meaningful efforts to share knowledge around RCS evaluations through initiatives like ESSENCE on Health Research. Drawing on that data, these studies promise to add to that dialogue and accelerate progress toward developing robust evaluations of health RCS.