What Makes a Great Teacher Great?

The LA Times has provoked the wrath of local teachers by posting a database of teacher performance based on a metric called “value added,” a measurement that supposedly controls for the quality of incoming students, English proficiency, individual behavior problems, and so forth. (Q&A here.) But a new EPI report says that value-added sucks anyway:

One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only predict from 4% to 16% of the variation in such ratings in the following year. Thus, a teacher who appears to be very ineffective in one year might have a dramatically different result the following year. The same dramatic fluctuations were found for teachers ranked at the bottom in the first year of analysis.

But, and this is an enormous caveat, everything else we currently use is worse. A teacher’s years of experience, their education credentials, their certification status, the prestige of their college or their college GPA, even in-class observations. None of these measures does as good of a job at predicting a student’s academic growth as a teacher’s value-added score. Yet, we continue to use these poor proxies for quality at the same we have such passionate fights about measures of actual performance.

Still, it remains true that virtually everyone agrees that value added is “not all that great” (Carey’s conclusion). So what should we do instead? Carey says we just don’t know. We have lots of minimum standards for teaching quality (no uncertified teachers, no violence, no classroom drunks, no overuse of mimeographed worksheets, etc.) but we don’t know much about what makes a teacher stellar:

What kind of teaching is as good as mimeographed worksheets are bad? We don’t really know. The qualifications-and-competence mindset doesn’t allow us to know. We can’t see it, and so gradually we allow policies and institutions and organizational cultures to evolve that pretend it doesn’t exist.

….There’s a natural tendency to proceed from here to the conclusion that we need to intensively study [great teachers like LA’s Zenaida Tan] so we can help others be more like them. And we should, this will be valuable research. But we ought not expect it will produce a new list of qualifications and competencies to which every teacher must adhere. Just as there are many different kinds of great writers and lawyers and artists, so too does great teaching come in all manner of variations. This should be seen as entirely positive for the teaching profession. The jobs worth having — and worth paying for — are those that can’t be wholly reduced to definable rules.

Yet the union that purports to represent Tan has done nothing but oppose the creation of the only measures that accurately identify her value as a professional. In doing so, it helps depress the public understanding of all teachers as professionals. If the LA Times hadn’t performed these value-added calculations and published them, who would have? How long do great teachers have to wait to be recognized? How long are they going to be held hostage to a mindset that pretends they don’t exist?

This is no surprise, I guess, since we have so little idea of what makes someone great at any profession. What makes a product manager great? Or a CEO? Or a magazine editor? Or a blogger? No one knows. If you go to the business section of your local Barnes & Noble you can find a hundred books with a hundred different vague and unhelpful answers based on little more than the author’s instincts. Talk to business professors and you’ll get some different answers, but probably not ones that are an awful lot more reliable.

But the problem with teachers is that assessing their performance isn’t just hard, it’s even harder than any of those other professions. Product managers interact closely with a huge number of people who can all provide input about how good they are. CEOs have to produce sales and earnings. Magazine editors and bloggers need readers.

But teachers, by definition, work alone in a classroom, and they’re usually observed only briefly and by one person. And their output — well-educated students — is almost impossible to measure. If I had to invent a profession where performance would be hard to measure with any accuracy or reliability, it would end up looking a lot like teaching.

So this means we end up using things like value-added, even though we know they’re not very good. What other choice do we have, after all? Sara Mead recommends greater reliance on “validated and reliable observational tools, such as the Classroom Assessment Scoring System (CLASS), that look at teacher classroom behaviors and measure the extent to which teachers are implementing behaviors linked to improved student outcomes,” and that sounds like a good idea to me. But even if this works, it will take years or decades to produce usable results. What do we do in the meantime?

The criticisms of value-added seem compelling. At the same time, if a teacher scores poorly (or well) year after year, surely that tells us something? At some point, we either have to use this data or else give up on standardized testing completely. It just doesn’t make sense to keep using it if we don’t bother taking the results seriously.

Looking for news you can trust?

Subscribe to our free newsletters.

DOES IT FEEL LIKE POLITICS IS AT A BREAKING POINT?

It sure feels that way to me, and here at Mother Jones, we’ve been thinking a lot about what journalism needs to do differently, and how we can have the biggest impact.

We kept coming back to one word: corruption. Democracy and the rule of law being undermined by those with wealth and power for their own gain. So we're launching an ambitious Mother Jones Corruption Project to do deep, time-intensive reporting on systemic corruption, and asking the MoJo community to help crowdfund it.

We aim to hire, build a team, and give them the time and space needed to understand how we got here and how we might get out. We want to dig into the forces and decisions that have allowed massive conflicts of interest, influence peddling, and win-at-all-costs politics to flourish.

It's unlike anything we've done, and we have seed funding to get started, but we're looking to raise $500,000 from readers by July when we'll be making key budgeting decisions—and the more resources we have by then, the deeper we can dig. If our plan sounds good to you, please help kickstart it with a tax-deductible donation today.

Thanks for reading—whether or not you can pitch in today, or ever, I'm glad you're with us.

We Noticed You Have An Ad Blocker On.

ONE QUICK THING:
Did you see that Mother Jones is launching a new Corruption Project? Check it out, and if our plan makes sense to you, we hope you'll help us raise $500,000 and go all in.

ONE QUICK THING:
Did you see that Mother Jones is launching a new Corruption Project? Check it out, and if our plan makes sense to you, we hope you'll help us raise $500,000 and go all in.

THE MOTHER JONES CORRUPTION PROJECT
We're crowdfunding to hire and build a new beat focused on systemic corruption—investigating how democracy and the rule of law are being undermined by those with wealth and power. Read why we believe this is what the moment demands, and please help fund it with a tax-deductible donation today.

THE MOTHER JONES CORRUPTION PROJECT
We're crowdfunding to hire and build a new beat focused on systemic corruption—investigating how democracy is being undermined by those with wealth and power. Read more, and please help fund it with a tax-deductible donation today.