How Effective Is Your School District? A New Measure Shows Where Students Learn the Most

Students in gym class at Mildred I. Lavizzo Elementary School in Chicago. “Whatever kids come in here, we know we can grow them,” the school’s principal, Tracey Stelly, says.Lyndon French for The New York Times

CHICAGO — In the Chicago Public Schools system, enrollment has been declining, the budget is seldom enough, and three in four children come from low-income homes, a profile that would seemingly consign the district to low expectations. But students here appear to be learning faster than those in almost every other school system in the country, according to new data from researchers at Stanford.

The data, based on some 300 million elementary-school test scores across more than 11,000 school districts, tweaks conventional wisdom in many ways. Some urban and Southern districts are doing better than data typically suggests. Some wealthy ones don’t look that effective. Many poor school systems do.

This picture, and Chicago’s place in it, defy how we typically think about wealth and education in America. It’s true that children in prosperous districts tend to test well, while children in poorer districts on average score lower. But in this analysis, which measures how scores grow as student cohorts move through school, the Stanford researcher Sean Reardon argues that it’s possible to separate some of the advantages of socioeconomics from what’s actually happening in schools.

In Chicago, third graders collectively test below the second-grade level on reading and math. But this data shows that over the next five years, they receive the equivalent of six years of education. By the eighth grade, their scores have nearly caught up to the national average:

Test scores for 3rd graders in 2,000 large school districts.

← Poorer districts

Richer districts →

By comparison, children in the Milwaukee Public Schools test at similarly low rates in the third grade but advance more slowly, leaving them even further behind by the eighth grade. In Maryland’s Anne Arundel County, third graders test above the national average. But growth there lags behind Chicago, where the poverty rate is about five times higher.

Across the country, this analysis shows, the wealth of a district tells us little about the effectiveness of its schools.

Change in test scores between 3rd grade and 8th grade

← Poorer districts

Richer districts →

“One question we’ve been asking ourselves is: Do urban public school systems simply reflect the poverty of the kids in the schools, or do they overcome those effects to any degree?” said Michael Casserly, the executive director of the Council of the Great City Schools, which represents large urban districts.

This new data shows that many do overcome them. It also suggests that states that rate schools and select which ones to reward or shutter based on average test scores are using the wrong metric, Mr. Reardon argues. And so are parents who rely on publicly available test scores to identify what they believe are the best school districts — and so the best places to live.

“Most people think there’s some signal in that,” Mr. Reardon, a professor of education, said of average test scores. “But it’s a pretty bad signal.”

Standardized tests, he acknowledges, are an incomplete measure of educational success, and some places like the state of Massachusetts have tried to include measures of growth in evaluating teachers and schools because of the limitations that Mr. Reardon points to.

Even the Chicago Teachers Union, however, warns that standardized tests don’t measure the richness of a curriculum, or whether students have access to librarians and college counselors. Mr. Reardon’s data also can’t detect when changes occur because students leave or enter a district between third and eighth grade. So demographic change may affect growth rates in a place like Anne Arundel, which has experienced an influx of children who are still learning English.

Educators have long debated whether it’s better to evaluate students and schools on proficiency levels or growth rates. Mr. Reardon’s data makes possible a national database of both. The No Child Left Behind Act of 2001 required states to conduct their own assessments in reading and math. This analysis converts those state scores, from 2009 to 2015, into a common standard measured in grade levels.

Districts with high growth are scattered across the country, in contrast with sharp geographic divisions on proficiency that show Northern schools ahead of those in the Deep South. School systems across Arizona and Tennessee that appear to test well below national averages are in fact overperforming in growth. Many predominantly minority districts where third graders start behind have high growth rates. But in New York City, where third graders test at the national average, slow growth puts them at a disadvantage later.

Highest growth rates

School district

Growth after 5 years

Lowest growth rates

School district

Growth after 5 years

Among the 200 largest school districts.

Even the fastest growth rates Mr. Reardon measures couldn’t completely close the proficiency gap that exists early on between typical poor and wealthy districts. That suggests that the most effective school systems alone can’t overcome all the disadvantages of poverty that accumulate before children even reach third grade and that shape the country’s racial achievement gaps.

There is promise, however, in a place like Chicago.

“Here’s the third-biggest school system in the country that’s dramatically outperforming not just the other big poor districts, but almost every district in the country, at scale,” Mr. Reardon said. If we understood what was causing that, in Chicago and other disadvantaged but high-growth districts, that might help reduce educational inequality, he said.

Lavizzo Elementary School in Chicago serves a student population that’s 98 percent black and 93 percent low income.Lyndon French for The New York Times

Even within this city, there’s broad disbelief in good news about the schools, in how they could succeed amid perpetual budget cuts, contentious school closings, rising crime and financial crisis.

But Mr. Reardon finds no evidence of inflated test scores in the district (by contrast, the recent cheating scandal in Atlanta is apparent in his data). Researchers at the University of Chicago and the University of Illinois at Chicago have also pointed to positive results for Chicago, relative to the rest of Illinois, and using other metrics.

“At some point, you’ve got to say, ‘O.K., this is getting to be an accurate picture,’ ” Mayor Rahm Emanuel said. The district has come far from 30 years ago, Mr. Emanuel notes, when Education Secretary William Bennett described the city’s schools as the worst in the country.

“I do wonder, if our students were not predominantly minority and poor, would people have the same level of skepticism?” said Janice Jackson, the district’s chief education officer since 2015 and a former teacher and principal in the system. “In public education we talk all the time about ‘beating the odds,’ about public education as ‘the great equalizer.’ But when we see it happen, we question it.”

Computer class at Lavizzo Elementary School on the far South Side of Chicago.Lyndon French for The New York Times

Mr. Reardon’s data shows that every demographic group within the district is growing at rates well above the national average, with Hispanic students outpacing whites. But while growth is broadly distributed, the pattern in Chicago and across the country means that black-white achievement gaps aren’t narrowing much even in the districts with the strongest growth.

On the city’s far South Side, scores have risen at Mildred I. Lavizzo Elementary School, which serves a student population that’s nearly 98 percent black and 93 percent low income. Several homes across the street are boarded up, and the area has lost population and jobs. Inside the school, the halls are decorated with emblems of other places: college banners, foreign flags, clocks that tell the time in Nairobi and Dublin.

Tracey Stelly, the principal since 2009, has brought in every enhancement she can find. The school uses an International Baccalaureate curriculum. The students read the Junior Great Books. The school hosts a community farmer’s market. Outside groups lead choir classes and organized games at recess.

“Whatever kids come in here, we know we can grow them,” Ms. Stelly said. She peered into the gymnasium one afternoon this fall while the fifth graders were dancing with their teachers to celebrate a schoolwide fund-raising project. “When kids come in the building,” she said, “they know, ‘This is where I belong.’ ”

At Lavizzo, the district’s emphasis on data and performance tracking is also conveyed to students in a manner Ms. Stelly hopes will inspire competition while remaining playful. One first-floor bulletin board updates the school’s attendance targets. Another records goals that students have set for their standardized test scores.

Across the district, data about attendance and grades is being used to identify the students likely to need extra attention. And the district has emphasized the role of more autonomous principals in improving instruction, an element of reform that Mr. Emanuel said is underappreciated nationally in debates that more often focus on teachers.

Students at Chicago’s Lavizzo Elementary School learn an International Baccalaureate curriculum.Lyndon French for The New York Times

The mayor has pushed other changes, including a longer school day and expanded pre-K, but those policies have shifted too recently to explain all the gains in Mr. Reardon’s data. Mr. Casserly suggests that Chicago and other large urban districts have been focused for years on the quieter work of defining what “grade level” actually means and how to get children there.

Between all these changes, it’s hard to untangle what’s been most effective, said Elaine Allensworth, who leads an education research consortium at the University of Chicago that works with the district. But she is confident the results are real.

“I go into schools now and I see places that are very different from what I saw 15 years ago,” she said. “It’s much more collaborative among teachers and data-focused, and focused on students.”

Compare your school district with its neighbors

Use the search box here or elsewhere on the page to see how your local school district compares with others in its county or nearby.

How compares with other nearby school districts in the state

District

Growth after 5 years

Nat. pct.

Median inc.

National percentiles are based on school districts with reliable data.

Recess at Lavizzo Elementary School in Chicago.Lyndon French for The New York Times

The richer and poorer axis on the charts reflects an index of socioeconomic status. It includes median family income; the percent of adults with a bachelor’s degree or higher; the poverty rate; the unemployment rate; the SNAP eligibility rate; and the percent of families headed by a single parent in a district. The data for each district includes charter schools located in its area.