Through the lens of social science, eduwonkette takes a serious, if sometimes irreverent, look at some of the most contentious education policy debates. (Find eduwonkette's complete archives prior to Jan. 6, 2008 here.)

Lessons for Performance Pay from the Financial Crisis?

This fall, we've heard a lot about how short-term pay incentives on Wall Street encouraged traders to take huge risks, and ultimately ushered us into our current financial mess. Ask the folks at Lehman Brothers - the decisions that maximize profits in the short-term don't always pan out in the long-term.

It's curious that at the same time, journalists and talking heads have pushed performance pay for teachers onto center stage. Proponents of performance pay often want to use one year of test score data in order to pass out bonuses - in other words, reward the attainment of short-term goals. But as Jay Mathews noted in his article this morning, the kind of growth we're after is what makes a student perform well in college, in the workplace, and in life.

There are good reasons to believe that the instructional strategies that produce short-term growth may not always serve students well in the long-term. So I suspect that there are at least two types of high value-added schools or teachers - those whose effects persist, and others who just produce short-term gains. Might we learn something from the financial crisis and hold some of that performance pay in escrow? Or revamp our accountability systems so they aren't focused solely on short-term gains?

And a teacher quality bonus:
Robert Pondiscio offers a smart response to the Mathews article.

Categories:

11 Comments

I have a colleague who argues that incentives are fine, but that the wrong things are being incentivized; that what we really care about is whether or not people go to college, etc. -- not how much their test scores go up in one year. This sounds similar, and I think it's a valid point.

That said, I find it odd that a year is considered short-term. In terms of responding to incentives, I think a year is a pretty long time. Imagine if a child's allowance was based on whether or not they did the dishes for a year rather than a week -- it would be pretty hard to keep them on track.

Paying teachers based on their students' test scores is problematic and unfair at so many levels that it doesn't really make sense to pursue a way to make it work. For one thing, how you make it work for an elementary music teacher? Or a preschool teacher working with developmentally disabled students? Another problem is that teachers are human. They will quickly figure out which areas in the curriculum will be measured and focus on those areas to the detriment of others.

A better approach would be to borrow from modern baseball statistics. (Sabermetrics) Basically, the idea is to measure players based not on the outcome of the performance (wins, batting average, RBIs) since those are subject to luck and uncontrolled variables. Rather, players are measured based on what they actually do: hit line drives, field balls within their expected range, pitch strikes at variables speeds, etc.

The way this would work in teaching is actually pretty simple. We have a vast body of research that tells us what teachers should be doing if they expect their students to learn. If we want to pay teachers for their performance, we watch them teach and measure the extent to which they do those things that research tells us will produce results. One practical way to do this already exists: National Board Certification.

Yesterday (11/17/2008), Randi Weingarten touted her plan to create a performance pay system for New York. She said: "Student achievement already has improved so significantly in 128 of those schools that their staffs are getting bonuses." She went on to say: "...[I]f an innovation is collaborative and fair, teachers will embrace it -- and it will succeed."

But there isn't much evidence that the system actually awards achievement or that it is fair. Consider both the principal bonuses and the school-wide bonuses.

One kind of bonus in this plan is given to principals. Principals at the top 20% of schools, based on overall Progress Report score, will each get a $7,000 bonus. There were 58 schools in the top 20% (Overall Score = 73.7 or better). Of those, 48 were excellent in a very clear way. They had better than average attendance. The average attendance in NY high schools is about 85.6%. About half of the schools in the district have less than that. Of the winners, 48 of the 58 had 85.6% or better attendance. To the winners, congratulations. Enjoy the $650,000. We didn't need a performance pay system or the Progress Reports to figure out how to award the money for attendance, but given the city's chronic attendance problems, the money is well worth the results. Perhaps these principals can tell us how they did it.

As for the school-wide bonuses, who knows how they were determined. Here are four of the biggest payout schools and their actual grades.

Think of how lopsided this year's grades were: almost all As and Bs. Above is a total of $2,157,000 in school-wide bonuses and not a single score of A on any part of the Progress Report. These schools didn't even get many points on the extra credit added to the scores (Gompers = 1, Columbus = 0, Boys and Girls = 1, Newtown = 3). What is the magic formula for earning this money? Just like the principal bonuses for attendance, the school-wide bonuses apparently have little to do with the Progress Report scores.

Unfortunately, National Board Certification has been shown to be a sham. Board certified teachers are not more effective than those who are not board certified. In fact, student achievement actually decreases while teachers are going through the board certification process. My students routinely do much better than the school average on the state tests. Last year my school implemented value added testing, and my students made the highest reading gains of any students at any grade level in our school. My class is larger than the others. I am given the most difficult students because it is assumed that I can handle them. How is it fair that I receive nothing for this? Value added assessments should not be rejected just because they are not absolutely perfect. Surely they are more fair than the current system of denying the most effective teachers any reward for their achievements.

A question for you, asked in earnest: Why is it "assumed" that you can handle the larger class and the most difficult students? Were your administrators aware of your abilities before the data showed up on your test scores?

My administrators are willing to give me the more difficult load because they can see that my class is calm and focused. Beyond that, I am evaluated by a silly checklist of things that waste an inordinate amount of time while doing little to improve student learning. It's funny, but what my principal likes most about my classroom, the students are relatively quiet and on task, is what I question the most about my teaching style. I have seen a video of a Japanese class that looked like a near riot by American standards, but it was clear that the kids were really learning. If I were judged by a value added standard, I would feel more free to try teaching methods that might displease my administrators (at least in the short term), but that might pay off in terms of student achievement. Value added assessments may not be perfect, but they are a vast improvement over the highly subjective process of administrator evaluations.

Judy: Two things:
1. You’re mistaken about the research. The NRC has recently released a study showing that NBCTs actually do have a positive, measurable impact on student achievement. This is on top of scores of other studies showing the same thing.

2. It sounds like you’re a very effective teacher. There’s no reason in the world why you shouldn’t be paid more than an ineffective teacher. The problem with value-added merit pay, as I noted earlier, is that it’s problematic and unfair. You know as well as I do that once we start paying for high test scores, we’re going to get high test scores. And little else. Furthermore, I can’t imagine a merit pay system that would be fair for all teachers. Think about the broad spectrum of teaching jobs: drama, PE, art, wood shop, English, kindergarten, third grade, science, preschool… Trying to institute a value-added merit pay system that would work fairly for all of these folks would be horrible complicated.

In Washington DC, WTU, the teacher's union co-funded bonuses to all members of the staffs of seven schools which gained the most, year on year, in test score performance. Serious $, as shown below.* http://tinyurl.com/6nvvwx
Well, I was saddened that an ECE school of 165 students, preK-3 was one of the winners. Sad, that principles of ECE, at what had been a demonstration center for the application of NAEYC standards, might have been diluted with test prep, even as I was happy for fact that the sizable bonus was shared by all.
Well, it turns out that only 3rd graders were tested, a Good Thing. And there was just one 3rd grade class and teacher, who almost singlehandedly secured the following award distribution for her colleagues.(By perhaps getting her 20 kids to answer a total of 50 more questions correctly than their predecessors had a year before?) (Whether other teachers kicked back a portion of their winnings to her or to her favorite charity is not known. )

*Teachers who work at these schools received TEAM awards of $8,000, with assistant principals receiving $9,000 awards and principals receiving $10,000 awards. All other instructional support personnel, including librarians, counselors and social service workers received awards of $4,000 each. Paraprofessionals, support staff and custodians each received $2,000 awards.

Incredulous: I read your post and find it hard to believe that VP's in improved schools earn a bigger bonus than the teachers. When does a VP have a bigger impact on math or English achievement than the math or English teacher? That's always the problem with these bonus systems - it's hard to decide who's responsible for the score improvements (if anyone). If I were a teacher at that school, I'd be irked that the VP's got a bigger bonus than I did. Then again, you could argue: Why is the kindergarten teacher getting a bonus for the third grade students' improvement?