From Stanford University, Stanford, California, and National Bureau of Economic Research, Cambridge, Massachusetts.

Acknowledgment: The authors thank Karen Stanecki of UNAIDS for help with providing and clarifying the data used in this study and Grant Miller, PhD, and Kanaka Shetty, MD, MS, of the Center for Health Policy at Stanford University for methodologic contributions.

Grant Support: By a training grant from the Agency for Healthcare Research and Quality (Dr. Bendavid) and the National Institute on Aging (Dr. Bhattacharya).

Potential Financial Conflicts of Interest: None disclosed.

Reproducible Research Statement:Study protocol: Not available. Statistical code: Available from Dr. Bendavid (e-mail, ebd@stanford.edu). Data set: The HIV epidemiologic data are available from UNAIDS (www.unaids.org). The complete data set is available from Dr. Bendavid (e-mail, ebd@stanford.edu).

Abstract

Background:

Since 2003, the President's Emergency Plan for AIDS Relief (PEPFAR) has been the most ambitious initiative to address the global HIV epidemic. However, the effect of PEPFAR on HIV-related outcomes is unknown.

Objective:

To assess the effect of PEPFAR on HIV-related deaths, the number of people living with HIV, and HIV prevalence in sub-Saharan Africa.

Design:

Comparison of trends before and after the initiation of PEPFAR's activities.

Setting:

12 African focus countries and 29 control countries with a generalized HIV epidemic from 1997 to 2007 (451 country-year observations).

Intervention:

A 5-year, $15 billion program for HIV treatment, prevention, and care that started in late 2003.

Measurements:

HIV-related deaths, the number of people living with HIV, and HIV prevalence.

Results:

Between 2004 and 2007, the difference in the annual change in the number of HIV-related deaths was 10.5% lower in the focus countries than in the control countries (P = 0.001). The difference in trends between the groups before 2003 was not significant. The annual growth in the number of people living with HIV was 3.7% slower in the focus countries than in the control countries from 1997 to 2002 (P = 0.05), but during PEPFAR's activities, the difference was no longer significant. The difference in the change in HIV prevalence did not significantly differ throughout the study period. These estimates were stable after sensitivity analysis.

Limitation:

The selection of the focus countries was not random, which limits the generalizability of the results.

Conclusion:

After 4 years of PEPFAR activity, HIV-related deaths decreased in sub-Saharan African focus countries compared with control countries, but trends in adult prevalence did not differ. Assessment of epidemiologic effectiveness should be part of PEPFAR's evaluation programs.

Primary Funding Source:

Agency for Healthcare Research and Quality.

Editors’ Notes

Context

The effects of the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) are unknown.

Contribution

Using data obtained from the United Nations, the authors detected a decrease in the number of HIV-related deaths in African countries receiving U.S. assistance after PEPFAR was introduced.

Caution

Many factors other than introduction of PEPFAR could have contributed to the observed findings.

Implication

The PEPFAR program seems to be having some favorable effects in African countries receiving AIDS assistance.

—The Editors

Join the dialogue on health care reform. Comment on the perspectives published in Annals and offer ideas of your own. All thoughtful voices should be heard.

Alleviating the burden of HIV in sub-Saharan Africa is one of the great challenges of our time (1). Infection with HIV is the leading cause of death in Africa, and it is responsible for a reduction in life expectancy in many countries (2). Available resources to prevent and treat HIV in less developed countries have expanded more than 5-fold since the Declaration of Commitment on HIV/AIDS, adopted by the United Nations General Assembly in 2001 (3, 4), but the expansion has not been evenly distributed. Since 2003, the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) provided the majority of its initial $18.8 billion budget to 15 focus countries—12 of them in Africa—for HIV and AIDS prevention, treatment, and care (5, 6). Although the criteria for selecting focus countries were not explicit, they were related to the burden of disease, the focus countries' governmental commitment to fighting HIV, administrative capacity, and a willingness to partner with the U.S. government. Nearly half of PEPFAR's resources were spent on antiretroviral drugs and treatment infrastructure, and about one fifth was spent on prevention programs, of which one third was earmarked for abstinence-only programs. Through its various activities, PEPFAR aimed to support the provision of life-saving antiretrovirals to 2 million people and prevent 7 million HIV infections in the focus countries within 5 years. Of the 3 major funders of HIV and AIDS programs (PEPFAR; The Global Fund to Fight AIDS, Tuberculosis, and Malaria; and the World Bank), PEPFAR is unique in its distinctive approaches and disproportionate funding of a few countries (7–10).

In July 2008, the U.S. Senate reauthorized PEPFAR with a $48 billion budget for the next 5 years, including a broader emphasis on strengthening health systems (11). Leadership in the battle against HIV has been one of the United States' most important legacies in Africa (12, 13). However, despite the substantial financial commitment and the important role PEPFAR plays abroad, no quantitative evaluation of the program's outcomes has been completed. The original legislation mandated a short-term evaluation that the Institute of Medicine completed in 2007 (14). The report scrutinized the ability of PEPFAR to meet its targets for delivery of prevention, treatment, and care services in the focus countries, and it found that within 2 years, PEPFAR supported expansion of HIV and AIDS activities in the focus countries; however, it did not evaluate health-related outcomes, such as HIV mortality, incidence, or prevalence.

The number of human lives affected and the financial stakes make it essential to assess the impact of PEPFAR's investment in Africa. Although the full impact of PEPFAR may not be felt for years, an ongoing evaluation of programmatic outcomes is central to the direction of future policies. Therefore, we quantitatively evaluated HIV-related outcomes in PEPFAR focus countries compared with other countries in sub-Saharan Africa with a generalized HIV epidemic.

Methods

Country Selection

All countries of central, east, west, and south Africa and the island nations of Cape Verde, Comoros, Madagascar, Mauritius, São Tomé and Príncipe, and Seychelles were eligible for this analysis. We excluded countries without epidemiologic data and those in which the HIV epidemic was not generalized. “Generalized epidemic” was defined as HIV prevalence of more than 1% in antenatal clinics and a predominantly heterosexual mode of transmission (9, 15). We examined the countries that PEPFAR selected as focus countries as the intervention group and designated all other sub-Saharan African countries with a generalized HIV epidemic as the control group.

Data Sources

The joint United Nations Programme on HIV/AIDS (UNAIDS) has monitored the Declaration of Commitment on HIV/AIDS by working with individual countries to measure the global epidemiology of HIV and AIDS (16). We used country and year epidemiologic data obtained through UNAIDS from 1997 to 2007 as the outcome variables for this study. The UNAIDS determines the prevalence trends by using sentinel and population-based surveys. The prevalence estimates, vital registry data, and model-based calculations are then used to determine mortality and the number of people living with HIV. All the estimates are given uncertainty bounds that depend on the quality of the primary data and the strength of the assumptions used in the estimation process (17).

We obtained data on Global Fund allocation of resources, population structure, governance indicators, life expectancy, and per capita gross domestic product from publicly available databases (10, 18–20).

Study Periods and Outcomes

We chose 2 study periods for this analysis: an early period, from 1997 to 2002, before PEPFAR began, and a late period, from 2004 to 2007, during PEPFAR's activities. We excluded 2003, the watershed year when PEPFAR's operations were getting organized. Three outcomes were examined: HIV prevalence among adults 15 to 49 years of age, the number of deaths due to HIV or AIDS, and the number of adults living with HIV. We used these indicators as outcomes because they reflect the most consistent measures for cross-country comparisons available over time. The indicators are publicly available (21).

Statistical Analysis

We compared trends of the epidemic outcomes between the focus and control countries during the early and late periods. We tested the hypothesis that the epidemic would show greater improvement (or less worsening) in the focus countries than in the control countries during the late period, after the onset of PEPFAR's activities. We included the early period to separate the effects of PEPFAR from the natural course of the epidemic and any between-country differences; that is, we wanted to compare not only the relative trends after PEPFAR began but also any changes in the relative trends before and during PEPFAR.

We analyzed the trends by using a fixed-effects model for longitudinal data with fixed time and country effects (22). We clustered by country and calculated robust standard errors because of the inherent bias in serially correlated data (23). Year and PEPFAR designation as a focus country, and an interaction term between them, were included as predictors in the unadjusted model. Except for prevalence rates, the outcomes we examined depend on the size of the population. To account for that, we examined the percentage change in the outcomes of interest by using log transformations of the outcomes (24). For prevalence rates, we report the difference between the focus and control countries of the change in prevalence rather than the difference in percentage change. We report the interaction term coefficient, which estimates the difference in the trends between the focus and control countries. For example, a coefficient of −2% in the number of deaths from 2004 to 2007 suggests that during that period, the percentage change in the number of deaths was 2% less in the focus countries than in the control countries (which means that deaths increased more slowly or decreased more rapidly, depending on the overall trend). We used Stata software, version 9.2 (StataCorp, College Station, Texas), for all analyses and Stata's xtreg command for the main regression.

In adjusted analyses, we accounted for baseline HIV prevalence in 1997, population size, life expectancy, per capita gross domestic product, the amount of funding from the Global Fund for HIV per capita, and 4 measures of governance from the World Bank's Worldwide Governance Indicators: control of corruption, political stability, rule of law, and government effectiveness (20). The indicators measure the quality of governance in more than 200 countries on the basis of 35 data sources from 32 organizations. They have been available since 1996 and are widely used in development research (25, 26). We also changed the model structure to allow for random year effects. Because none of these additional analyses changed the direction or significance of the relative effect of PEPFAR, we report the results of the unadjusted analyses.

To account for uncertainty in the primary data and model estimates of the UNAIDS data, we did 2 sensitivity analyses (27). First, we performed a Monte Carlo process where the outcome variable was drawn randomly from a uniform distribution of values within the uncertainty bounds for each country-year observation, and the analysis was repeated 1000 times. We also performed a sensitivity analysis where we randomly selected groups of “focus” countries and ran the analysis 1000 times to verify our analyses were not biased in favor of finding significant differences (23).

Role of the Funding Source

This work was supported in part by a training grant from the Agency for Healthcare Research and Quality. The funding agency had no part in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript.

Results

Sample Characteristics

We examined 12 PEPFAR focus countries and 29 control countries from sub-Saharan Africa for the analysis. Tables 1 and 2 show the selected countries and their baseline characteristics, respectively. The focus countries had a significantly higher average population (37 million people vs. 11.7 million people; P = 0.009), but there was no difference in the life expectancy of men or women. The per capita disbursements from the Global Fund were slightly more generous in the control countries, but the annual per capita gross domestic product was significantly lower in those countries ($1935 vs. $4094). The focus countries ranked higher than the control countries on all governance indicators, and significantly higher in government effectiveness, at the time of PEPFAR's start. The adult HIV prevalence was higher in the focus countries throughout both study periods (P ≤ 0.051).

Table 1. Focus and Control Countries

Table 1. Focus and Control Countries

Outcome Measures

Number of Deaths From HIV or AIDS

The number of deaths from HIV or AIDS increased in both groups during the early period but decreased in the late period in the focus countries and leveled off in the control countries (Figure 1). Before 2003, the annual percentage increase in the number of deaths due to HIV was 3.5% higher in the control countries, but that difference was not significant (P = 0.22). During the late period, however, the death toll from HIV decreased much more rapidly in the focus countries than in the control countries. During this period, the difference in the percentage change was 10.5% lower in the focus countries (P = 0.001) (Table 3). Figure 2 shows that the difference in death rates between the groups started after 2003 and was most pronounced between 2005 and 2006, after nearly 3 years of PEPFAR activities.

Figure 1.

Longitudinal epidemic trends of the study outcomes.

Data are presented as means (95% CIs).

Table 3. Differences in Outcomes Between the Focus and Control Countries During the Early and Late Study Periods

Table 3. Differences in Outcomes Between the Focus and Control Countries During the Early and Late Study Periods

Figure 2.

Differences in outcomes between focus and control countries over time.

Each data point represents the difference in the percentage change from the previous year to the current. That is, each point represents (Xt − Xt−1/Xt−1)focus − (Xt − Xt−1/Xt−1)control for outcome X in year t. Negative numbers mean that the outcome decreased faster (or increased more slowly) in the focus countries. Deflections away from zero suggest that the differences became more pronounced, whereas deflections toward zero mean that the trends were becoming more similar. The dotted lines represent the transition period around 2003, when the President's Emergency Plan for AIDS Relief was being organized.

Number of People Living With HIV or AIDS

The number of people living with HIV or AIDS increased in the focus and control countries throughout both study periods (Figure 1). The annual percentage change in the number was lower by 3.7% in the focus countries during the early period (P = 0.051) (Table 3). In the late period, during PEPFAR's activities, the annual percentage increase in the number of people living with HIV or AIDS was slower in both groups of countries. However, the difference (1.7% lower in the focus countries) was no longer significant (P = 0.124). That is, between the early and late periods, the focus countries experienced a relative acceleration in the number of people living with HIV or AIDS compared with the control countries. Figure 2 shows that the difference in the annual percentage change was diminishing (getting less negative) after 2003, but the difference in the number of people living with HIV/AIDS was negative (the increase was slower in the focus countries) before 2003.

Adult Prevalence of HIV

The HIV prevalence in both groups peaked by 2002 (Figure 1). Before 2003, the prevalence increased faster in the focus countries than the control countries (0.05% faster annually), but this difference was not significant. During PEPFAR's activities, the difference in prevalence change between the groups was nearly 0% (P = 0.95), as both groups of countries experienced a gradual decline of about 0.1% annually in the HIV prevalence in the adult population. Figure 2 shows that the difference in the adult prevalence diminished throughout the early period and remained stable around 0% after 2003. Appendix Tables 1 to 6 provide additional details on the regression models.

Sensitivity Analysis

To assess the stability of our analyses, we used the uncertainty bounds provided by UNAIDS to select values for the outcome measures. We used a uniform distribution to draw a value for the outcome variables, avoiding assumptions about the true value. We then used the randomly picked value for each country-year as the value for the regression. This process was repeated 1000 times, and we collected the mean of the coefficients and P values. Using this procedure, the difference in the annual percentage change in the deaths due to HIV or AIDS in the focus countries was −2.0% during the early period (P = 0.29) and −9.7% in the late period (P = 0.0198) compared with the control countries. A significant difference in deaths from HIV was observed in 90.7% of the iterations in the late period but only 3.7% of the iterations in the early period. Appendix Table 7 shows full results of the sensitivity analysis.

Our second procedure for verifying the stability of our results, in which we randomly selected groups of countries to serve as “focus” countries for the analysis, showed that any significant differences, especially in the number of deaths, disappeared.

Discussion

We examined the effect of PEPFAR by comparing changes in 3 outcome measures between the focus countries and other countries with a generalized HIV epidemic in sub-Saharan Africa, before and after PEPFAR began. Our results suggest that after 4 years of activity, PEPFAR was associated with a decrease in deaths due to HIV or AIDS and may be linked to a relative increase in the number of people living with HIV or AIDS. We see no evidence that PEPFAR was associated with changes in adult HIV prevalence in the focus countries compared with other sub-Saharan African countries.

The reduction in HIV-related deaths is probably the result of improved treatment and care of HIV-infected persons in the focus countries, especially the greater availability of highly active antiretroviral therapy. Antiretroviral therapy coverage increased disproportionately in the focus countries during the past 5 years (9), and nearly half of PEPFAR's expenditures are dedicated to purchasing antiretrovirals, constructing treatment infrastructure, and providing antiretroviral services (28). Around the time of PEPFAR's launch, other governmental and multinational organizations, most notably the Global Fund, also scaled up their activities to combat HIV, which may have contributed to the decline in HIV deaths seen across the continent. However, the added infusion of funding for antiretrovirals in the focus countries made an appreciable impact on the deaths from HIV and indicates the power of antiretrovirals to improve survival in a relatively short period.

We observe a relative acceleration in the number of people living with HIV or AIDS in the focus countries relative to the control countries during PEPFAR's activities. Although long-term goals may target reductions in the size of infected populations, this increase probably reflects the decreasing death rate and may have several public health spillover benefits. For example, infected adults who live longer may be able to support their children and dependent elderly family members, reducing the burden of orphans and elderly care.

Changes in prevalence are complex and depend on the rate of new infections, deaths due to HIV or AIDS, and changes in the size of the population. We see no evidence that prevalence trends in the focus countries differed from those in the control countries during PEPFAR's activities. To effect a reduction in HIV prevalence, the combined effect of reduced HIV incidence and increased population size must offset the reduction in deaths from HIV or AIDS. Although it may be too early to observe these changes, it is important to follow this trend for several reasons: measurement of prevalence is standardized in many countries, longitudinal trends through sentinel sites are widely available, and it is a key determinant of infection risk (29). A reduction in prevalence that may be attributable to PEPFAR would be a consequential accomplishment for the next 5 years of PEPFAR.

As the number of people receiving antiretroviral therapy and the deaths averted in the focus countries continue to increase, the cost of providing treatment is expected to increase as well. Projections of financial resources needed to sustain the treatment scale-up suggest that even with PEPFAR's greater commitment, the gap between the available funds and those needed will continue to increase unless the incidence of HIV in Africa is substantially reduced (3). Striking the right balance between treatment and prevention with insufficient resources for the burden of the epidemic is a major challenge for comprehensive care programs, such as PEPFAR.

Monitoring impact will have important implications for the future of PEPFAR, as well as other organizations that are operating with poor information about the effectiveness and efficiency of their programs. In a recent report, the Institute of Medicine drafted general considerations for evaluating PEPFAR's impact, including HIV prevalence, mortality, and incidence, as well as broader metrics, such as system capacity, economic development, and health status (30). Incidence in particular has not been directly estimated in Africa, but measurement techniques are increasingly available (31). Impact evaluations are difficult, but rigorous methods adapted specifically to resource-limited regions, including randomization into program entry, are commonly used in other disciplines (32).

We evaluated the contribution of PEPFAR to the abatement of the epidemic in the focus countries, which has implications for the program's economic efficiency. By the end of 2007, PEPFAR spent more than $6 billion on HIV care, prevention, and treatment in the 12 focus countries examined in this study. In those countries, a reduction in the death rate of 10.5% implies that about 1.2 million deaths were averted because of PEPFAR's activities. This large benefit cost about $2450 per death averted, assuming that PEPFAR directed half of its budget toward treatment. This is a rough estimate, and it may change as the treatment infrastructure and supply chains become more established, but it could allow an evaluation of the program's efficiency. A formal cost-effectiveness analysis will also allow a comparison with other interventions for HIV in Africa (33).

Our study has several limitations. First, we used UNAIDS data that are derived in part through mathematical models. We dealt with this limitation by doing a sensitivity analysis using the uncertainty bounds, which take into account the imprecision in both the primary data and the model estimates. We used a resampling procedure that is agnostic about the exact values of outcomes within the uncertainty bounds and strengthens the results of the primary analysis. It is also possible that the data in the focus countries are more reliable than those in the control countries. This is unlikely to change the results, however, as PEPFAR's support for monitoring and evaluation programs was minor and this effect is likely to be small.

Second, we performed cross-country comparisons between groups of countries that were not picked at random and had significant baseline differences. We addressed this in part by controlling for observable potential sources of bias, such as population, gross domestic product, aid for HIV from other sources, and governance indicators. Because the focus countries were not selected randomly and we cannot fully observe the differences between the groups, our measured effects may be specific to the countries and years of the study—that is, we cannot fully generalize the results to other countries or other periods. If PEPFAR had chosen a different set of focus countries or operated at a different time, we may have observed a different impact.

Third, the observed difference in deaths after 2004 may have resulted from a difference in the phase of the epidemics in the study countries. That is, if the epidemic in the focus countries was more mature than in the control countries, then the observed relative reduction in deaths could occur without any intervention or program. However, the lack of difference in HIV deaths before 2003 argues against this possibility as an explanation of the results.

Although the Institute of Medicine scrutinized PEPFAR's operations, it did not evaluate outcomes that are central to prevention and treatment efforts regarding the epidemic. The commitment of funds by the U.S. government is commendable, but it is crucial to ascertain that PEPFAR is effective and that the investment in this program is cost-effective. As PEPFAR enters its next funding period, evaluating outcomes will highlight the areas that are successful and those that are not making an appreciable impact. Our analysis shows the success of PEPFAR in averting HIV-related deaths in a relatively short period. It also underscores the importance of a continued outcome-based evaluation of this essential and expensive intervention.

Office of the United States Global AIDS Coordinator. The Power of Partnerships: The President's Emergency Plan for AIDS Relief Third Annual Report to Congress. Accessed at www.pepfar.gov/documents/organization/81019.pdf on 17 March 2009.

Figure 1.

Longitudinal epidemic trends of the study outcomes.

Data are presented as means (95% CIs).

Figure 2.

Differences in outcomes between focus and control countries over time.

Each data point represents the difference in the percentage change from the previous year to the current. That is, each point represents (Xt − Xt−1/Xt−1)focus − (Xt − Xt−1/Xt−1)control for outcome X in year t. Negative numbers mean that the outcome decreased faster (or increased more slowly) in the focus countries. Deflections away from zero suggest that the differences became more pronounced, whereas deflections toward zero mean that the trends were becoming more similar. The dotted lines represent the transition period around 2003, when the President's Emergency Plan for AIDS Relief was being organized.

Table 3. Differences in Outcomes Between the Focus and Control Countries During the Early and Late Study Periods

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.