What Works Clearinghouse Quick Review of the Article "Are High-Quality Schools Enough to Close the Achievement Gap? Evidence from a Social Experiment in Harlem"

What Works Clearinghouse

This study examined the effects on academic achievement of offering students enrollment in the Promise Academy charter middle school. The school is sponsored by the Harlem Children's Zone[R], which combines reform-minded charter schools with a web of community services designed to provide a positive and supportive social environment outside of school. The study analyzed data on about 470 New York City students who applied for enrollment in 2005 and 2006 as entering sixth graders. The number of applicants exceeded the school's capacity, so enrollment offers were granted by random lottery. The study measured effects by comparing the outcomes of students who were selected in the lottery and offered enrollment in the school to students who were not selected in the lottery. Student outcomes were measured in sixth, seventh, and eighth grades using standardized statewide math and English language arts (ELA) tests. Students offered enrollment in the school had higher math test scores in sixth, seventh, and eighth grades than the students not offered enrollment. By the time they were tested in eighth grade, the effect size for the math test was 0.55. The WWC interprets this as equivalent to moving a student from the 50th to the 71st percentile. The study authors found no statistically significant differences in ELA test scores in sixth or seventh grade, but a positive effect was found on the eighth grade ELA test. The effect size was 0.19, which the WWC interprets as equivalent to moving a student from the 50th to the 58th percentile. The research described in this report is consistent with WWC evidence standards. The study is equivalent to a randomized controlled trial because the groups of students contrasted in the study were formed by random lottery. [The following study is reviewed in this quick review: Dobbie, W., & Fryer, R. G., Jr., (2009). "Are high-quality schools enough to close the achievement gap? Evidence from a social experiment in Harlem." (NBER Working Paper No. 15473). Cambridge, MA: National Bureau of Economic Research. See ED507097.]