Don't rush to link teacher evaluation to student achievement

By
Valerie Strauss

My guest is Susan H. Fuhrman, president of the National Academy of Education, and president of Teachers College, Columbia University. A longer version of this piece appeared in Education Week. This is part of an occasional series by Teachers College faculty, who will write about reforms proposed by the Education Department in “A Blueprint for Reform,” the Obama administration’s vision of how to rewrite the federal law commonly known as No Child Left Behind.

By Susan H. Fuhrman
The Obama administration’s “Blueprint for Reform” of the Elementary and Secondary Education Act encourages states to evaluate teachers based in part on their students’ scores on standardized tests. It is likely, then, that linking teacher evaluation to student achievement will play a significant role in upcoming policy initiatives and renewal of ESEA, the federal law more commonly known as No Child Left Behind.

In order to ensure fairness, most plans to evaluate teachers based on their students’ performance attempt to control for differences among students and other factors that are beyond the teachers’ control. They use the so-called “value-added” approach.

Recently, the National Research Council and the National Academy of Education jointly issued a report on value-added approaches, based on findings from a November 2008 workshop funded by the Carnegie Corporation of New York. According to the report, value-added models refer to sophisticated statistical techniques that measure student growth. They use one or more years of prior student test scores, as well as other background data, to adjust for pre-existing differences among students when evaluating student test performance.

Carnegie workshop participants were very cautious about using value-added measures for high-stakes decisions, such as salary and tenure rulings, which affect individual teachers. One reason is that a large number of teachers have students who are not given standardized tests.

These include teachers in the earliest grades and those in subjects like art, music, and social studies. It would be most unfortunate if attempts to improve teacher accountability ended up increasing one of the most criticized aspects of current accountability systems—namely the over-reliance on standardized tests.

Moreover, even in the cases where tests already exist, such as for teachers of reading and mathematics in grades 3-8, value-added approaches raise significant concerns.

Recent research suggests that they give an accurate picture of teacher-related gains in achievement only if students are randomly assigned to teachers. But if, for example, administrators systematically assign struggling students to the “best” teachers (as may be the case in many schools) or to new, inexperienced teachers (as is the case in many other schools), those teachers’ measured gains relative to those of their colleagues will likely suffer.

Other concerns about value-added models include:

*Many tests cover different content from one grade to the next, so score gains do not have the same meaning across grades. Many state assessments, in fact, are not scaled to measure grade-to-grade growth or to make growth comparisons.

*Value-added estimates for a teacher can fluctuate for a variety of reasons, many not necessarily related to actual effectiveness. For example, high turnover of students throughout the year can affect the gains students make on achievement tests; and if the class size is small, the scores of only a few students can affect the size of the gains. These kinds of errors can be reduced—but not eliminated—if administrators take several years of teacher performance into account when making important decisions.

*Additional factors can affect student performance, including the efforts of other teachers involved with a student, learning support outside of school (tutoring, parental help, and the like), and other family and societal factors that might influence student achievement.

The lesson of the NRC-NAEd report is that even though value-added methodologies offer a number of advantages over other approaches that consider test-score data in a vacuum, policymakers need to move carefully in adopting any approach—value-added or otherwise—in making important decisions about individual teachers.

Value-added approaches hold great promise, but there is a need to develop better tests (and other thoughtful measures of student learning) and better measures of teacher practice to use along with test scores, so they are not the sole factor used to evaluate teacher effectiveness.

Last year, I had a student who moved out of state from our district one full month before state assessments were given. The answer sheets were already printed by that point, and had to be turned in with the completed tests of the other students, as per state regulations. When the results were released at the beginning of this year, her score (not suprisingly A ZERO for all subjects) was counted in with my results! Nothing was done about it, even though both my principal and myself appealed it. Well, this year, I had a child who moved at lunch on the first day of testing--missing out on the other 3 days of testing. When I asked, I was told that the test would be scored AS IS. So sorry!
As always, the devil is in the details.

I can't begin to imagine how using test scores as a major factor in evaluating teachers could be done fairly. What happens when students are continually entering and leaving schools between September and June? In the last school where I taught, one-third of the children changed during the year and attendance at the school was poor.

When these standardized tests are weighed so heavily, my concern is cheating. It would be so easy to change students' answers. If they are going to use these tests, the schools should not be the ones administering them. Many schools live or die as a result of these tests. There is a local charter high school here in Sac that has had dramatic improvement in test scores since the once public high school turned charter. What's very interesting is that the SAT scores for the school are plummeting at an alarming rate. I find that correlation interesting...

Now this, Valerie, is high quality article written by an intelligent individual. It's pragmatic and, for me, demonstrates a balanced perspective on a controversial issue. More pieces like this could elevate the status of this column exponentially.

You and Susan Furhman are to be applauded on this effort.

I would add one caveat to her observations; the test results should be used primarily for improvement of instruction, at least initially, until some of the problems can be teased out of the process. Variables such as assuring random placements and teachers evaluated who don't have their students tested can be left to the mutual devices of the LEA and their respective teacher union to negotiate under their collective bargaining agreements.