How Do We Keep Schools from Screwing Up?

Warning

A new study shows that families act on insufficient information when it comes to figuring out where to enroll their children.

GAIL CORNWALL 11:00 AM ET EDUCATION

… A new working paper titled “Do Parents Value School Effectiveness?” suggests that parents similarly opt for schools with the most impressive graduates rather than figuring out which ones actually teach best. The study joins a body of research looking critically at what it means for a school to be successful.

Take the work of Erin Pahlke, for example. The assistant professor of psychology at Whitman College saw research showing that girls who attend school only with other girls tend to do better in math and science. The trick, she said, is that those studies didn’t analyze “differences in the students coming into the schools.” As it turns out, those who end up in same-sex schools tend to be wealthier, start out with more skills, and have parents who are more proactive than students who attend co-ed institutions. In a 2014 meta-analysis, Pahlke and her colleagues reviewed the studies and found when examining schools with the same type of students and same level of resources—rather than “comparing [those at] the public co-ed school to [their counterparts at] the fancy private school that’s single-sex down the road”—there isn’t any difference in how the students perform academically. Single-sex schooling also hasn’t been shown to offer a bump in girls’ attitudes toward math and science or change how they think about themselves. In other words, it often looks like single-sex schools are doing a better job educating kids, but they aren’t. It’s just that their graduates are people who were going to do well at any school. They’re running on high-octane gas.

So too are high schools widely thought to be “life-changing”—the elite ones that students must test into. In a 2014 Econometrica paper titled “The Elite Illusion,” the economists Atila Abdulkadiroğlu, Joshua Angrist, and Parag Pathak wrote that while students who attend extremely competitive public schools like Stuyvesant High School in New York City clearly excel, that may not mean the schools provide an education that’s superior to their less competitive counterparts. The researchers looked at a group of borderline kids, the last few eighth-graders who made the cut-off to go to an elite school and the first few who didn’t; that meant there was little if any academic difference between them when they started their freshman year. If a school like Stuyvesant were more effective—that is, taught more material and produced better outcomes—than the less competitive public school, the economists would expect to see a difference in how those kids performed academically four years later. But when the researchers analyzed indicators of success, such as AP exam scores and state standardized tests, they saw no difference between the borderline kids who got to attend Stuyvesant and the borderline ones who didn’t. And yet, said Pathak, a professor of microeconomics at MIT, “these are massively oversubscribed schools. People would give an arm and a leg to send their child to a school like Stuyvesant.”

On the other hand, there’s a Noah Baumbach movie from about 10 years ago in which all the other characters think Jack Black’s character is a complete idiot until he happens to mention he went to Stuyvesant, causing them to re-evaluate his intelligence upward.

Anyway, it’s been apparent since the 1966 Coleman Report that it is fairly difficult to find overwhelming evidence of any schools dramatically improving student performance. I’m not saying it hasn’t been done, just that the default is that the outputs of most schools correlate with the inputs in terms of student quality more than they correlate with inputs like budgets.

But, perhaps it is time for social science researchers to look for the mirror image situation: schools that do much worse than their inputs would suggest. If it’s hard to do much better, maybe we should focus more on not doing much worse?