Bulletin of the World Health Organization 2010;88:379-385. doi: 10.2471/BLT.09.070607

Introduction

The availability of well trained and motivated health workers in underserved areas will improve access to essential health services to achieve the health-related United Nations' Millennium Development Goals within the framework of a primary health care renewal.1–3 Yet there are stark imbalances in the geographical distribution of health workers, both in developed and developing countries. Approximately one half of the world's population lives in rural areas but these areas are served by only 38% of the total nursing workforce and by less than 25% of the total physicians’ workforce.2 At the country level, imbalances in the distribution of health workers are even more prominent.4,5

In recent years, there has been increased interest from both researchers and policy-makers to identify and implement effective solutions to address the shortages of health workers in remote and rural areas.6–11 In response to this increased interest and perceived need, the World Health Organization has recently launched a programme of work on “Increasing access to health workers in remote and rural areas through improved retention.” The programme aims to expand the knowledge base in this domain and to provide evidence-based global recommendations to address this problem, while at the same time to provide technical cooperation to countries that need to address this problem.12

As part of this programme of work, this paper builds on and expands earlier work on assessing the evidence on effectiveness of interventions to increase access to health workers in rural and remote areas.13 This paper expands the original search to focus mainly on studies that evaluated such interventions, and attempts to analyse the impact of such interventions on certain dimensions of health workforce and health systems performance. It also discusses the quality of the evidence from evaluation studies and identifies the evidence gaps in this domain. It is expected that evaluations would give policy-makers additional information with regards to the effectiveness and applicability of various interventions in their own context.14

Conceptual framework

The analysis in this paper is based on the assumption that the final result of having health workers in remote and rural areas depends on two inter-linked aspects8: (i) the factors that influence the decision or choice of health workers to relocate to, stay in or leave those areas, and (ii) the extent to which health system policies and interventions respond to these factors. These responses are usually grouped into four main categories: education, regulatory, financial and personal and professional support interventions.6–10

There is a wealth of descriptive literature highlighting the extent of geographical imbalances and deficits in health personnel in rural and remote areas,4,5 or reporting on the factors that influence health workers’ preferences or choices with regards to practicing in remote and rural areas.15,16 There are also studies that describe or recommend potential interventions, without analysing the effects of these interventions.17–19 However, there is very limited research on comprehensive evaluations of specific retention strategies. We set out to conduct a review of this type of study with the aim of further informing the methodology in conducting evaluations of rural health workforce retention strategies.

Methods

We conducted an extensive review of the literature that reported on evaluations of interventions to increase the availability of health workers in remote and rural areas. Electronic searches were conducted in August and September 2009 in PubMed, the Cochrane database, Embase® and LILACS. Reference lists of the retrieved studies were also searched to complement the final list of articles. Further evidence was gathered from experts in the field of human resources for health, and from grey literature, through searches in Google, the Human Resources for Health Global Resource Centre and various web sites of government ministries.

We included articles that were published between 1995 and September 2009 that reported on the results/effects of an intervention to increase availability of health workers in rural or remote areas, from both developed and developing countries, and covering all types of health workers, and including a clear description of the study design and methods used. We excluded studies that described the issue of misdistribution, studies reporting on surveys of factors influencing choices of practice, without a direct link to a specific intervention, and studies that were only describing a potential intervention. We also excluded news, editorials, policy briefs and commentaries as they did not report on a specific evaluation.

A total of 14 746 studies were retrieved from the electronic searches. Two reviewers screened all titles and abstracts found in the search. The reviewers independently judged the titles and abstracts according to the above specified inclusion and exclusion criteria. All titles and abstracts judged as eligible for this global review were then retrieved and thoroughly reviewed in their entirety. Upon full review of the texts, further studies were deemed as not eligible for the inclusion criteria and were discarded (Fig. 1 provides the search diagram and reasons for exclusion). Both reviewers were in agreement over the final selection of studies included in the global review.

Fig. 1. Flowchart depicting the selection process for studies included in the review

Results

Twenty-seven studies met our inclusion and exclusion criteria and were included in the final analysis. The results reported by these studies are presented against a framework for monitoring and evaluating retention interventions.20 This framework proposes four dimensions on which various policy interventions can have a direct effect: attractiveness of rural/remote areas for students and/or health works, recruitment/deployment, retention, and health workforce or health system performance. These dimensions follow logic by which it is expected that the creation of “demand” (attractiveness) will be followed by recruitment and retention, which in turn will have a positive effect on improving the availability of health workers, and the quality of health services, thus improving population health outcomes in the long term. For each of these dimensions, specific indicators were reported. A summary of the indicators, methods and results reported in the evaluation studies included in our review is included (available at: http://www.who.int/hrh/resources/evaluation_tables). In addition, for each study included in the review we also specified the category and the subtype of intervention, i.e. education, regulatory, financial, or professional and personal support.

Attractiveness

Twelve studies reported on interventions that have attracted students towards working in rural and remote areas, the majority of them being education interventions. Studies from developed countries have consistently shown that health professionals from a rural background are more likely to practise in rural areas, clinical rotations in a rural setting may influence medical students’ subsequent decision to work in an underserved area, and appropriate educational preparation for rural service, including adapting curricula to include rural health issues, creates more interest to work in these areas.21–24 Additionally, several studies consistently showed that rurally oriented medical education programmes influenced subsequent choices of graduates to practise in rural areas21,23,25–28 (Appendix A, available at: http://www.who.int/hrh/resources/evaluation_tables).

Intentions to relocate to, stay in or leave rural areas have also been influenced by other categories of interventions. One study on financial incentives in South Africa found that the allowance had positively influenced the future plans of health workers, particularly nurses, to work in a rural area.29 Positive changes in the intentions to stay in rural areas were also reported in a continuing professional development programme,30 a programme using financial incentives for long-serving rural doctors in Australia,31 a personal support programme for rural general practitioners in Australia,32 a programme providing rural clinical experiences to medical students, and a scholarship scheme for medical students, the latter two in the United States of America (USA)33,34

Recruitment

Seven studies have reported the effects of interventions on the number of graduates or health workers actually recruited in underserved areas following their implementation. Two multifaceted educational programmes in the USA have consistently shown a steady increase in the number of their graduates recruited to work in rural areas.26–28,35,36 A bonding scheme for post-graduate students in Australia also showed an increase in the percentage of workers practising in rural areas.37 For other types of interventions, the size of effect, expressed by the proportion or number of workers recruited in rural areas as a result of the scheme, is relatively small. For example, a compulsory service in South Africa reported less than 25% of physicians recruited to rural facilities,38 and a financial incentive scheme in the Niger, targeted at doctors, pharmacists and dental surgeons, reported an increase of 42–44% in recruited workers.39 Finally, a programme aiming to support physicians setting up their practice in rural areas in Mali has reported that more than 100 young doctors were recruited in those areas over a 10-year period40 (Appendix B, available at: http://www.who.int/hrh/resources/evaluation_tables).

Retention

Various indicators were used to measure effects on retention: length of service, proportion of health workers staying in rural areas, survival rates, turn-over rates and settlement rates. Two studies reported on length of service, one showing a mean duration of 15 months for physicians engaged in a network of rural academic family practice, and the other reporting an average of 4 years in the “medicalization of rural areas” programme in Mali.40,41 Other studies reported only on the proportion of health workers remaining in the rural areas. This varied from a 20% retention rate in a bonding scheme in USA, through to 86% for a financial incentives programme in Australia.31,42 A more comprehensive analysis was provided by Matsumoto et al., who reported on the effects of a rurally located medical school coupled with a compulsory service scheme in Japan. Almost 70% of the graduates of Jichi medical school remained in their home prefectures for at least 6 years after their obligatory service.43 Finally, another way of measuring retention was to determine the “survival rates” of health workers, using the Kaplan–Meyer method for survival analysis. For example, Rabinowitz et al. found that more than two-thirds of graduates from the Physician Shortage Area Programme in the USA remained in family medicine in the same rural area after 11–16 years28 (Appendix C, available at: http://www.who.int/hrh/resources/evaluation_tables).

Impact

Ten studies have reported on the effects of rural retention interventions on the performance of health workers, or on health services and communities. Two studies reported that certain competencies of health workers improved as a result of retention interventions,30,38 while improved job satisfaction was reported in five evaluations of retention strategies.32,34,39,44,45 Finally, four studies reported on the effects of rural retention interventions on service delivery and communities, in terms of improved quality of care, reduced referrals and reduced waiting times33,38,46,47 (Appendix D, available at: http://www.who.int/hrh/resources/evaluation_tables).

It was noted that most of the interventions had multiple effects on the continuum from attraction through to recruitment, retention and, finally, health workforce or health systems performance. Fig. 2 depicts the mapping of the interventions included in our review against these proposed dimensions, and the indicators that were reported by these studies. This is an important finding, as policy-makers need to be clear from the start about the expected results of the chosen intervention and should be able to make choices on the relative importance of the various expected results. Knowing the expected results is also key for monitoring and evaluating these interventions.

The shading indicates the relationship between the interventions and the indicators and dimensions.

Discussion

We assessed the methodological quality of the studies included in our review using traditional criteria for evidence-based clinical medicine: study design, consistency, precision, directness, likelihood of publication bias and the magnitude of effect.48

Study design

We found no randomized controlled trials analysing the effects of retention interventions in rural or remote areas. Five longitudinal cohort studies reported on the effects of multifaceted education programmes26–28,49 and a financial incentive scheme.29 Three retrospective cohort studies reported on the effects of a medical education programme,33 a loan repayment scheme50, and a compulsory service programme.43 Five studies used a before-and-after design: two looked at clinical rural placements,23,25 and one in each of the compulsory service,32 financial incentives39 and personal or professional support programmes.44 Only one study used a control group in comparing the results of a financial incentives programme between rural physicians enrolled in the scheme and those who were not.34 The remaining twelve studies used a cross-sectional observational design and did not report any baseline against which to compare the observed changes.

Reporting results

In terms of methods used, the majority of the studies employed mixed methods, using both quantitative and qualitative methods to describe the complexity of the effects of such interventions. However, many studies relied upon surveys and questionnaires, for which inferences may be difficult to make, and did not always account for design, biases and sampling. Interestingly, only one study used census data to analyse the location of health workers before and after the intervention.39

There was great variability in the reported outcomes, specifically on retention, with some studies considering retention rates as the number or the proportion of health workers remaining in the area, while others providing more comprehensive measures that accounted for both the number of workers and the duration of their stay in months or years. For some studies, there was also a misalignment of outcomes reported in relation to the retention scheme they were evaluating. For example, Fisher et al. found that community health aides had longer retention rates than doctors or nurses, but concede that it was not possible in their study to link these outcomes to the telemedicine intervention they originally wished to evaluate.49 The magnitude of effect therefore is difficult to judge because of these inconsistencies in reporting across the studies.

Publication bias

Our findings confirm other observations,6,8–10 which showed a skewed geographical distribution of studies that analysed the effectiveness of rural retention interventions. Most evidence comes from high-income countries, such as Australia, Canada, Japan, New Zealand and the USA, with very few studies originating from developing countries in Africa, Latin America or south-eastern Asia, and no evaluations from the eastern Mediterranean region. This is also interesting when compared with our previous findings on the regional representations of descriptive research, where many studies have been conducted in sub-Saharan Africa describing the extent of the problem or the factors influencing choices and preferences of location.13 Also, there is a wealth of anecdotal evidence from many regions, which may not get published due to language barriers. There is a clear need to expand evaluation and operational research efforts in medium- and low-income countries from these regions.

Directness and relevance

The final result of having health workers in remote and rural areas is dependent upon both the factors that influence health workers preferences for rural or remote areas, and the health system’s response to these factors.8 In other words, the intervention has to respond to the factors that health workers value in choosing to work in these areas.

The findings showed that rural retention interventions are rarely implemented as a result of an analysis of the preferences or choices of health workers to practise in these areas. Two exceptions are worth mentioning: Dunbabin et al. provide a comprehensive situation analysis before moving on to evaluate the policy response of postgraduate medical placements in rural areas in Australia;37 and in the Niger, a study analysing the motivations of health workers to work in rural areas preceded the development of a financial allowance scheme, which was subsequently evaluated after two years of implementation.39 Given their importance in the design and implementation of retention strategies, it is critical to have a very good understanding of these factors. More refined research methods are needed to identify these factors and to gauge their relative weight in the final decision of health workers.51

Content gaps

The majority of evaluations were conducted for educational programmes and, to a certain extent, regulatory interventions such as compulsory service or bonding schemes. Only four studies were found in both the financial incentives and support programmes categories. This gap is crucial for two reasons. First, policy-makers are inclined to adopt financial incentives programmes almost as a “first-aid” measure when confronted with shortages of health workers in rural or remote areas, without enough knowledge about the effectiveness of such interventions or about their sustainability in the long run. Second, there is an apparent reluctance to adopt professional and personal support measures, even though these factors consistently top the surveys analysing choices and preferences for work in these areas.8 It is important to have an in-depth understanding of the effects of such interventions if policy-makers are to adequately address the expectations of health workers in rural areas.

Finally, a significant gap is that almost all studies have evaluated programmes targeted at physicians or graduates of medical schools. Only three studies reported on other categories of health workers: two rural clinical placements programmes for pharmacy students in New Zealand and nursing students in Australia, and one programme on financial incentives for doctors, pharmacists and dental surgeons in the Niger.23,25,39 This gap needs to be urgently addressed by the research community, as rural practice requires all categories of health workers and, more importantly, effective health teams to deliver good quality services.

Conclusion

We focused our analysis on studies that have evaluated the impact of policy interventions to increase availability of health workers in rural areas, because evaluation studies can provide useful insights for policy-makers who are faced with health-worker shortages in those areas.14 This allowed us to confirm previous findings on the effectiveness of such interventions6,8–10 but, more importantly, to provide a clear account of the outcome measures, methods and indicators used, to better inform future evaluation research.

We identified certain gaps in the published evidence on evaluation research in this field: there is an overall bias towards physician-targeted programmes, and towards developed English-speaking countries. More efforts should be made to evaluate programmes targeted at other types of health workers and, in particular, health teams and programmes from developing countries. With regards to the latter, efforts should also be made to strengthen human resources information systems so that sound analysis can be conducted.

In terms of scope, the majority of evaluations were for education programmes, with little information about financial incentive programmes, which are frequently used by policy-makers, and about professional and personal support programmes, which are most valued by health workers.

In terms of relevance, there is frequently a lack of coherence between the proposed retention strategy and the factors that matter for health workers in their choice for location. A situation analysis, including a survey of factors that influence choices of location of health workers in rural areas, should be mandatory as a basis for selecting the most appropriate category of intervention. Finally, in terms of methodology, using a control group is a powerful way to demonstrate the results of a certain intervention; therefore comparative studies, as well as cohort studies or controlled experimental studies should be increasingly encouraged and appropriately funded.

The added value of this study is to provide a clear and systematic account of methodological challenges and research gaps in conducting evaluation research in the field of rural health workforce retention. It is hoped that this will help future evaluations in choosing the study design and the methods appropriate for this type of research. However, although assessing the methodological quality of such studies is necessary, we acknowledge that more needs to be considered by policy-makers when deciding the type of interventions to use in their own context. While traditional criteria to assess the methodological quality have proven useful in clinical medicine, when used for public health and health systems interventions they risk yielding only low quality or very low quality evidence, which by itself can deter policy-makers to take any action on these interventions. Therefore, additional information needs to be provided to policy-makers with regards to the relevance of the interventions to certain contexts and situations and the mechanisms by which certain interventions have worked in some contexts but not in others.52 Better informing the choice of appropriate interventions to address the geographical imbalances of health workers, and assessing progress on implementing these interventions, will inevitably contribute to the broader goals of universal coverage and to the achievement of the health-related Millennium Development Goals.

Acknowledgements

We thank members of the World Health Organization's core expert group on “Increasing access to health workers in remote and rural areas through improved retention” and Mario Dal Poz of the Health Workforce Information and Governance Team, Department of Human Resources for Health, World Health Organization, for comments on earlier drafts of this paper.

Funding:

Financial support for literature searches in languages other than English was provided by the Health Workforce Migration and Retention Unit, Department of Human Resources for Health, Health Systems and Services, World Health Organization, Geneva.