Share this:

Last fall Josh McGee and I developed the Global Report Card (GRC) for the George W. Bush Institute. The GRC is a tool that allows people to compare the level of academic achievement in virtually every school district in the United States to the average for their state, the country, and a comparison group of 25 industrialized countries.

The Global Report Card received a fair amount of coverage when it was released, but keep your eyes out for an updated and improved version this upcoming fall. The results of the GRC are consistent with other international comparisons, including a series of pieces by Eric Hanushek, Paul Peterson, and Ludger Woessmann (the most recent of which can be found here). But the GRC goes a step further by allowing comparisons to be made at the school district level. GRC 2.0 will also have some new features and comparisons that people might find useful.

As impressive and amazing as these results by charter schools may be, it would be wrong to conclude from this that charter schools improve student achievement. The only way to know with confidence whether charters cause better outcomes is to look at randomized control trials (RCTs) in which students are assigned by lottery to attending a charter school or a traditional public school. RCTs are like medical experiments where some subjects by chance get the treatment and others by chance do not. Since the two groups are on average identical, any difference observed in later outcomes can be attributed to the “treatment,” and not to some pre-existing and uncontrolled difference. We demand this type of evidence before we approve any drug, but the evidence used to justify how our children are educated is usually nowhere near as rigorous.

Happily, we have four RCTs on the effects of charter schools that allow us to know something about the effects of charter schools with high confidence. Here is what we know: students in urban areas do significantly better in school if they attend a charter schools than if they attend a traditional public school. These academic benefits of urban charter schools are quite large. In Boston, a team of researchers from MIT, Harvard, Duke, and the University of Michigan, conducted a RCT and found: “The charter school effects reported here are therefore large enough to reduce the black-white reading gap in middle school by two-thirds.”

The same Stanford researcher conducted an RCT of charter schools in Chicago and found: “students in charter schools outperformed a comparable group of lotteried-out students who remained in regular Chicago public schools by 5 to 6 percentile points in math and about 5 percentile points in reading…. To put the gains in perspective, it may help to know that 5 to 6 percentile points is just under half of the gap between the average disadvantaged, minority student in Chicago public schools and the average middle-income, nonminority student in a suburban district.”

When you have four RCTs – studies meeting the gold standard of research design – and all four of them agree that charters are of enormous benefit to urban students, you would think everyone would agree that charters should be expanded and supported, at least in urban areas. If we found the equivalent of halving the black-white test score gap from RCTs from a new cancer drug, everyone would be jumping for joy – even if the benefits were found only for certain types of cancer.

Unfortunately, many people’s views on charter schools are heavily influenced by their political and financial interests rather than the most rigorous evidence. They don’t want to believe the findings of the four RCTs, so they simply ignore them or cite studies with inferior research designs in which we should have much less confidence.

Progress will be made in our application of research to charter school policies by encouraging everyone to focus on the most rigorous studies, of which we have several. To do that, supporters of charter schools also have to refrain from citing weaker evidence, which only serves to legitimize the use of inferior studies by charter opponents. As exciting as the outstanding performance of charter schools is in my own Global Report Card research, that evidence shouldn’t be used to endorse charter schools. Supporters don’t need to rely on the Global Report Card to make the case for charter schools because they have four gold-standard RCTs on their side. Opponents of charter schools have no equally rigorous evidence on their side. And that’s the point we should all be making.

Share this:

According to the Global Report Card that Josh McGee and I developed, tiny Waconda, Kansas is one of the top-performing school districts in the United States. Other than being the home to what residents claim is the world’s largest ball of twine (pictured above), one might not think that there was anything exceptional about this rural, farm community in north central Kansas.

But in 2007 the average student in Waconda performed better than 91% of students in our 25 country comparison group in math achievement. If we relocated Waconda to Finland, the average student in Waconda would outperform 88% of the students in Finland in math.

A reporter for Yahoo News was curious about what they were doing right in Waconda. Here is what she found:

So why are Waconda kids–65 percent of whom live in poverty–doing so well? And can other schools follow their lead?

The Waconda district comprises four small towns–Cawker City, Downs, Glen Elder and Tipton–and seven schools spread over 411 square miles. Most people in the area work in agriculture or in manufacturing.

The district’s superintendent of seven years, Jeff Travis, told Yahoo News that after years of high test scores, the community expects its students to excel. Most years, he added, no one drops out of high school. The district won 14 state Governor Achievement Awards and one national “Blue Ribbon Award School” over the past four years.

“It’s a tradition now, and they expect themselves to do well,” Travis said. “Like a ball team that continues to win because of a tradition, we have an academic tradition.”

Still, the community doesn’t quite seem to get how exceptional they are. “Everybody’s pretty happy [but] nobody understands how big a deal it is,” he said.

Travis says the students’ high level of achievement is even more extraordinary given that 65 percent of them qualify for free or reduced federal lunches, an indication that they live in poverty. High poverty schools are often dogged by low test scores and high dropout rates. Many educational observers indeed blame the nation’s sky-high child poverty level for the country’s comparatively low performance in math.

One theory Travis has is that Waconda school kids have no sense that they’re materially deprived. “North Central Kansas is rural, and urban poverty is kind of different [from] rural poverty,” he said. “A lot of our people don’t even understand that they’re living in poverty.” According to state data, most of the students are white, and no kids need English language learning classes.

About 10 percent of the students in the school district are foster kids, Travis says. “We just [have] a lot of adults that care about kids, so it’s been a popular thing for parents to take in foster children.”

But Josh adds a useful note of caution at the end of the reporter’s piece:

One of the Global Report Card’s authors, Josh McGee, says the small size of Waconda schools may have skewed the results slightly, since randomness has a greater impact on a smaller sample size. Most of the best-performing school districts in his ranking were small, and many of them were also made up of charter schools. You can read more about his methodology here.

We may not be able to generalize much from the success in Waconda, but it is a fun and impressive story.

Meanwhile, coverage of the Global Report Card continues to stream in. For an updated list of media for the Global Report Card (with links), click here.

The last blog post contained some criticisms about whether the assumptions for the analysis were reasonable. Josh McGee replied in the comment section of that post. And NCES Commissioner, Jack Buckley, told Education Week that “The methodology in this report is highly questionable.” This assessment is a little strange because what we did was similar to what the U.S. Department of Education has done in several past reports linking international test results to state NAEP results. (See for example this.) We just bring the results down to the district level. If ours is highly questionable, then the U.S. Department of Education’s own efforts must also be questionable.

(UPDATED 12-19-11)

Share this:

Education reform has really focused on improving the quality of education for our most disadvantaged students. This focus is not entirely without reason, since large, urban school districts serving low-income students are clearly dysfunctional.

But this nearly exclusive focus on improving the education of the poor has concealed the sub-par education being provided in many of our most affluent school districts. As the new article Josh McGee and I wrote for Education Next shows, suburban public school districts may look good when compared against their urban neighbors, but when compared with students in 25 other developed countries many affluent suburbs barely keep pace. That is, our best is often mediocre.

If the children of affluent suburbanites want to maintain their parents’ high standard of living, they need to be performing near the top relative to student overseas with whom they now have to compete for high-paying jobs in an increasingly globalized economy. Doing better than the kids in big city school districts should provide suburbanites with little comfort.

But this is precisely the comparison we encourage suburbanites to make. State accountability testing shows suburban districts doing better than the rest of the state, which consists largely of big urban districts. Policymakers and reformers talk endlessly about the “achievement gap,” highlighting how much worse low-income and minority students are doing. As Rick Hess recently noted, “our achievement gap mania” has stifled the innovation we need to improve education across the board.

It’s an old saying in public policy that “programs for the poor are poor programs.” The same is true in education. If we focus exclusively on improving the education in big cities we fail to engender the support education reform needs from suburban elites if it is to be successful. As long as suburbanites think that education reform is something for those poor kids in large urban districts, they will never fully commit to the kind and scale of reform that is really needed to improve things in big cities as well as everywhere else. They’re afraid to muck up what they think is a successful education system for their own children.

As our new Education Next piece shows, this suburban complacency is not well-founded. Suburbanites need education reform for the sake of their own children and not just for the poor kids in the big cities. If suburban elites commit to education reform for their own children,we may finally get improvement for low-income kids in the cities as well.

Student achievement in virtually every one of the nearly 14,000 public school districts in the United States compared to students overseas can be found at The Global Report Card’s interactive web site. With the support of the George W. Bush Institute, we’ve been able to provide this information so that everyone can look up their own and other districts to see that the need for education reform is not confined to big cities.