Teacher quality and test scores: Recent studies raise questions

As we reported earlier this week, the standoff continues between our state and the feds over the use of student test scores in teacher evaluations.

The U.S. Department of Education continues to insist that test scores should play some role in teacher evaluations. Washington lawmakers have refused to require school districts to do so and, as a result, lost the state’s waiver from the requirements of the federal No Child Left Behind Act.

This week, the feds refused Washington’s request to get back a piece of that waiver — the part that would have saved schools from having to send letters home saying they have failed — as most other schools in the nation have failed — to ensure that all students were proficient in reading and math this year.

So what about the substance of the argument? Are test scores a valid indicator of a teacher’s effectiveness?

People on both sides can point to research that supports their views.

Here are a few of the latest reports and studies, which raise a lot of questions:

1. In May, two researchers looked at whether the way some districts are using test scores to judge teacher performance is a good gauge of teaching quality. They found, to their surprise, that it doesn’t seem to be.

Morgan Polikoff and Andrew Porter looked at data about 327 fourth- and eighth-grade teachers in six school districts. They looked at the ratings teachers received from an analysis of their students’ test scores, often called value-added measures. Then they looked at other measures of quality, including how well teachers cover the material they were supposed to teach under their state’s learning standards.

Their bottom line: “What we’re left with is this concern that state tests really aren’t picking up the things that we think of as being good teaching,” Polikoff said in a video that accompanied the release of the study, which appeared in Educational Evaluation and Policy Analysis, a peer-reviewed journal of the American Educational Research Association.

“What these results, among other results, call into question is the sort of fixed and formulaic approach to teacher evaluation that is being promoted in a lot states right now,” he said.

(Their research was sponsored by the Bill & Melinda Gates Foundation, which is also the main sponsor of Education Lab.)

2. In April, the American Statistical Association urged caution in the use of value-added measures, saying most studies find that individual teachers account for, at most, 14 percent in the variation in test scores.

“The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences,” the association wrote.

3. Also in May, the Brookings Institution concluded that judging teacher quality by observing them in the classroom may not be any better than looking at student test scores — and may be worse.

They found that teachers who start the year with academically strong students tend to get better evaluations than teachers with weaker students.

“We should not tolerate a system that makes it hard for a teacher who doesn’t have top students to get a top rating,” the authors wrote.

Stories in the series

When tackling the topic of student discipline, some of the country’s toughest schools have done a turnaround. Instead of focusing on rules broken, they now ask kids to confront themselves. The result? Fewer suspensions and new perspective on the point of school itself. Read the story →

It stands to reason: Kick troubled students out of school and they often come back even worse. The Kent School District is trying to tackle this national problem by overhauling the way it handles discipline. But its answers spark even more questions. Read the story →

In an idea borrowed from college athletics, the University of Washington boosts promising engineering students — many of them women and minorities — with an extra year of academic work. Read the story →

Boosting the quality of preschool in Seattle could help children, and the city as a whole. A number of studies, including one from the ’60s, establish that potential. But there is no guarantee of success. Read the story →

Universal, free preschool in Tulsa, Okla., has produced results attracting national attention, and could be a blueprint for Seattle. But after 16 years the long-term outcomes raise almost as many questions as they answer. Read the story →

Communication failures both within Seattle Public Schools and with parents of children with disabilities continue to undermine the district’s efforts to fix longstanding problems in special education. Read the story →

A new focus on individualized advice and counseling, boosted by software tools, is helping hundreds more students earn degrees and certificates each year at Walla Walla Community College. Read the story →

The path to college often leaves disadvantaged students behind. Two unusual nonprofits, one based in Seattle, have helped vault thousands of low-income students onto university campuses. Read the story →

In an attempt to add depth to the curriculum in America's most popular advanced high-school courses, some local teachers threw out most of their lectures and replaced them with a series of projects. Results so far are encouraging. Read the story →

Western Washington University college students are working as mentors, tutors and role models for thousands of K-12 students in and around Bellingham. The goal: convince them that college should be part of their educational trajectory. Read the story →

Kent educators combed through transcripts and discovered 2,600 young people in their district without any kind of diploma or credential. Enter iGrad, a program linking dropouts with college, that has been flooded with kids who want a second chance. Read the story →

A community group in northwest Chicago has turned hundreds of hesitant parents into capable classroom helpers, role models and leaders by tapping into strengths many don't realize they have. Read the story →

Missing just a few days of class in sixth grade can predict whether you'll graduate from high school. That research powers a national anti-dropout effort that's making a difference at Seattle's Aki Kurose and Denny International middle schools. Read the story →

For years, students at White Center Heights Elementary logged some of the lowest test scores in King County. Then teachers tried something new, and the numbers soared by double-digits after just one year. So what happened, and could it be replicated elsewhere? Read the story →

About the authors

John Higgins is one of Education Lab's reporters. He was a Knight Science Journalism Fellow at Massachusetts Institute of Technology (MIT) from 2012 to 2013.

Katherine Long has been a reporter for The Seattle Times since 1990, focusing for the past three years on higher ed, with stories that have ranged from the complexities of prepaid tuition programs to nontraditional ways to earn a degree.

Claudia Rowe joined The Seattle Times’ reporting staff in 2013. She has written about education for The New York Times and the Seattle Post-Intelligencer, among other publications.

Leah Todd is an education reporter at The Times. She previously covered education for the Casper Star-Tribune in Wyoming.

Mike Siegel has been a news photographer at the Seattle Times since 1987. His photography was used in a series titled "Methadone and the Politics of Pain," which won a Pulitzer Prize in 2012 for investigative reporting.

Linda Shaw is The Times’ education editor. Previously, she covered public education as a reporter at The Seattle Times for more than two decades. Her coverage has won numerous national and local awards and honors.

Caitlin Moran is community engagement editor for Education Lab. She came to The Times from Patch, where she spent three years managing hyperlocal news websites on the Eastside.

About Solutions Journalism Network

The Education Lab project is being done with the support of the Solutions Journalism Network. SJN is a non-profit organization created to legitimize and spread the practice of solutions journalism: rigorous and compelling reporting about responses to social problems.