How Much Bang-for-the-Buck Do We Get from Education Interventions? A Look at the ChalleNGe Program

When kids graduate from high school with a diploma or earn a GED, both they and society benefit. Research shows that "high school dropouts" are more likely to earn low wages, be un- or underemployed, commit crimes, abuse drugs and alcohol, have children out of wedlock, and suffer poor health than individuals who successfully complete high school. This means that programs that intervene to keep kids in school can have significant benefits--assuming they actually work. But even if they yield benefits, program costs may outweigh those benefits.

For years, RAND researchers have assessed the cost-effectiveness of early childhood programs and showed that such programs could save the government money in the long run and produce benefits for society as a whole. Drawing on this expertise, they assessed the costs and benefits of a program serving high school dropouts: the ChalleNGe Program.

Staying the Course

About 25 percent of high school freshman fail to graduate from high school within four years. The ChalleNGe Program seeks to alter the course for such high school dropouts. It is an intensive residential and mentoring program at National Guard base training centers across the country that focuses on discipline, academic excellence, teamwork, physical fitness, leadership, and community service. It encourages participants to obtain a GED and seek further education and training or employment during a one-year post-residential phase in which attendees receive mentoring and other support and guidance.

It turns out that it works. A rigorous random assignment evaluation showed that the ChalleNGe Program has had positive effects on educational attainment and employment: Thirty-six months out, participants (compared with those not admitted) increased their GED attainment by 22 percentage points, traditional high school diploma attainment by 4 percentage points, some college attendance by 16 percentage points, vocational training and employment by 7 percentage points, and annual earnings by about 20 percent.

Totaling Up the Costs

Impressive as these benefits are, the bigger issue is how much it costs to get them. RAND researchers examined the evaluation results to try to answer that question. All told, program costs--including operating costs and opportunity costs for those involved with the program--ran $15,436 per participant.

The Other Side of the Ledger

On the benefits side, the program leads to greater educational attainment, which, in turn, helps drive higher labor market earnings. RAND researchers estimated that over his or her working life, the average ChalleNGe attendee (relative to the non-attendee) accrues an additional $43,514 in present discounted earnings.

While earnings are the largest component, the ChalleNGe program also yields $1,334 in social benefits per participant, because, down the road, attendees (relative to non-attendees) end up being less dependent on social welfare, committing fewer crimes, and increasing the amount of service they provide back to the community.

Coming Out Ahead

Even when we add in the costs of education (around $4,900), the estimated benefits far outweigh the estimated costs for each attendee. With benefits of $40,985 against costs of $15,436, the ChalleNGe program's net benefit per participant is an estimated $25,549, or $2.66 in benefits for every dollar expended. That means that the estimated return on investment in the ChalleNGe program is 166 percent.

Such results seem to demonstrate the effectiveness of the ChalleNGe program. Its estimated return on investment is considerably higher than that of many other rigorously evaluated programs that seek to alter the life course of disadvantaged youth and young adults.

INTERVIEW

Getting the Word Out About What Works

David S. Loughran is a RAND senior economist, an associate director of RAND Labor and Population, and a professor of economics at the Pardee RAND Graduate School. His research focuses on applied topics in labor economics, demography, and insurance. His current research includes an analysis of the causal effect of military service on labor supply and educational outcomes among socioeconomically disadvantaged youth. He received his Ph.D. in economics from the University of Maryland. Read more about David S. Loughran »

Finding that programs like ChalleNGe both work as intended and yield benefits should really help practitioners and policymakers.

Such results are very useful, but only if they are accessible to those who would most benefit from having them. For research organizations, the typical output of analyses is going to be a stand-alone technical report or, perhaps, a series of articles in the relevant journals for the field. Unfortunately, that means that valuable results can become buried or lost or, if found, can be very hard to interpret or understand.

That's where something like the Promising Practices Network or the What Works Clearinghouse would come in, right?

Yes, such clearinghouses are critical when it comes to helping practitioners and policymakers find or understand interventions. The PPN, which is managed by RAND, offers credible, evidence-based information on what works to improve the lives of children and families. The What Works Clearinghouse--an initiative of the U.S. Department of Education's Institute of Education Sciences--is more focused on educational interventions. But the intent is the same: They both translate research for decisionmakers trying to find out what does and doesn't work in a particular area, making it easier to interpret and apply the results.

So what shows up for something like ChalleNGe?

In the case of ChalleNGe, if you go on the PPN site, you will find the program listed under the heading of "programs that work." What you will see is a very accessible discussion of the results of the random assignment evaluation that was done--the gold standard for evaluations--as well as other information about participants and funding and other things. The cost-benefit results of our study have been added to the entry.

Such sites have other benefits as well.

That's true; there is no shortage of interventions out there, but not all of them have been shown to be effective. If you were to search the Internet for interventions in a particular area, the search would yield a slew of interventions that might sound very promising. But it can be very hard to tell whether such programs actually work. Sites like PPN or the What Works Clearinghouse do the heavy lifting for practitioners and policymakers and help them winnow out the effective from the ineffective programs.

Anything else?

Even among proven programs like ChalleNGe, there are differences. Clearinghouse sites allow practitioners and policymakers to compare across proven programs to see which might be the most appropriate choices. For example, in our cost-benefit analysis, we found that not only did ChalleNGe have benefits that outweighed its costs, but the estimated returns on investment were higher than they were for other similar proven programs that seek to alter the life course of disadvantaged youth and young adults, such as Job Corps, Big Brothers Big Sisters, and state welfare-to-work programs. Such comparative information can be very useful.

RELATED LINKS

The Promising Practices Network

The Promising Practices Network (PPN) is a user-friendly website operated by RAND that provides evidence-based information on child, youth, and family policy. Site content is organized around four main sections: Programs that Work, Research in Brief, Service Delivery, and Partner Pages.