Study: Outside of school factors play a larger role in test scores than teachers

T.J. Hamilton | MLive.comMegan Racette, fifth-grade teacher at Bowen Elementary School in Kentwood, near Grand Rapids, works with her students during class.

GRAND RAPIDS, MI – Outside of school factors play a larger role in standardized test performance than teachers, according to research released by a union-backed think tank.

The National Education Policy Center study summarized research looking at the effect of tests in relationship to linking student progress on the exams to teacher evaluations.

The report is the first in a series by William Mathis, the center’s managing director, said linking test performance to evaluations is problematic.

The connection between student performance and teacher rankings have been controversial in Michigan and other states, and was one of the factors that led to Chicago teachers going on strike for more than a week this month.

“Teacher quality is among the most important within-school factors affecting student achievement. However, research also suggests that teacher differences account for no more than about 15 percent of differences in students’ test score outcomes,” Mathis wrote.

William J. Mathis

“Other school factors such as class size reduction and adequate, focused funding are also research-based ways to improve education. Further, non-school factors, which are generally associated with parental education and wealth, are far more important determinants of students’ test scores.”

Mathis wrote there are three large problems with linking exams to evaluations, saying the “measurement error” is large, potentially leading to many teachers being incorrectly labeled as effective or ineffective.

He said that relevant test scores would not be available for most teachers since only certain grade levels and subject areas are tested.

Also, the incentives created by use of high-stakes tests could lead to “undesirable teaching practices” such as narrowing the curriculum or teaching to the test.

“Care must be taken in selecting or designing a balanced evaluation system,” he wrote. “Given the extensive range of activities, skills, and knowledge involved in teachers’ daily work, the system’s goals must be clear, explicit and reflect practitioner involvement. Effective teacher evaluation also requires an investment in sufficient numbers of qualified evaluators. Otherwise, the system will likely be irregular, uneven and ineffective.”

Mathis said any single measure of teaching will emphasize one important element at the expense of others. He said any system should use a variety of measures “to capture the complex nature of the art and science of teaching.”

Fourteen Michigan districts are piloting an evaluation plan that incorporate scores from a variety of exams and multiple classroom observations using four methods.

The goal is to have a statewide system in place by next school year.

Meanwhile, districts are using evaluation plans negotiated with their unions. An Education Trust-Midwest study of the state’s largest districts revealed 87.75 percent of teachers were deemed “effective,” and 11.60 percent were ranked higher, as “highly effective.” Just 0.65 percent of the teachers were deemed “ineffective” or “minimally effective.”

The University of Colorado-based NEPC is backed by the Ford Foundation, the Great Lakes Center, and the National Education Association, according to the organization's website. The Great Lakes group is funded by the National Education Association and the union branches in Illinois, Indiana, Michigan, Minnesota, Ohio and Wisconsin.