Dear Deborah,
A few weeks ago, Mike Rose [a professor at UCLA] posted a list of his New Year's resolutions. One of them was that we should "make do with fewer economists in education. These practitioners of the dismal science have flocked to education reform, though most know little about teaching and learning." Mike suggested that so few economists were able to give useful advice about the financial and housing markets that we should now be skeptical about expecting them "to change education for the better."

I agree with Mike.

It is astonishing to realize the extent to which education debates are now framed and dominated by economists, not by educators or sociologists or cognitive psychologists or anyone else who actually spends time in classrooms. My bookshelves are chock full of books that analyze the teaching of reading, science, history, and other subjects; books that examine the lives of children; books that discuss the art and craft of teaching; books about the history of educational philosophy and practice; books about how children learn.

Now such considerations seem antique. Now we are in an age of data-based decision-making, where economists rule. They tell us that nothing matters but performance, and performance can be quantified, and those who do the quantification need never enter a classroom or think about how children learn.

So the issue of our day is: How do we measure teacher effectiveness? Most of the studies by economists warn that there is a significant margin of error in "value-added assessment" (VAA) or "value-added modeling" (VAM). The basic idea of VAA is that teacher quality can be measured by the test-score gains of their students. Proponents of VAA see it as the best way to identify teachers who should get merit pay and teachers who should be fired. Critics say that the method is too flawed to use for high-stakes purposes such as these.

Last July, the U.S. Department of Education published a study by Mathematica Policy Research, which estimated that even with three years of data, there was an error rate of 25 percent. A few months ago, I signed onto a statement by a group of testing experts, which cautioned that such strategies were likely to misidentify which teachers were effective and which were ineffective, to promote teaching narrowly to the test, and to cause a narrowing of the curriculum.

None of these cautions has stemmed the tide of rating teachers by student test scores and releasing the ratings. Last year, the Los Angeles Timespublished an online database that rated 6,000 teachers as to their effectiveness (one of them, elementary school teacher Rigoberto Ruelas, committed suicide a few weeks later). New York City is poised to make a public release of the names and ratings of 12,000 teachers, if the courts give the go-ahead (in the first trial, a judge ruled that the data could be released even if it was inaccurate).

This trend did not just happen. It was encouraged by the Obama administration's Race to the Top, which urged states to develop quantitative measures of teacher effectiveness. Secretary of Education Arne Duncan has issued statements endorsing the publication of teachers' names and ratings, although few testing experts agree with this practice.

The bulk of studies warn about the inaccuracy and instability of these measures, but the Gates Foundation recently released a study called "Measures of Effective Teaching" (MET) that supports the use of VAA and VAM. As is customary for the Gates Foundation, it hired an impressive list of economists at institutions across the nation to give the gloss of authority to its work. Among its key findings was this one: "Teachers with high value-added on state tests tend to promote deeper conceptual understanding as well." Ah, said the proponents of measuring teacher quality by the rise and fall of student test scores, this study vindicates these methods and effectively counters all those cautionary warnings.

But now comes a re-analysis of the Gates study by University of California-Berkeley economist Jesse Rothstein, which says that the MET study reached the wrong conclusions and that its data demonstrate that VAA misidentifies which teachers are more effective and is not much better than a coin toss.

Even the claim that teachers whose students get high scores on state tests will also get high scores on tests of "deeper conceptual understanding" is flawed, writes Rothstein. The correlation between the two tests was actually modest: About 29 percent of the teachers in the bottom quintile on the basic skills tests were rated above average on the tests of reasoning and critical thinking; these are the teachers who would be fired if the Gates Foundation had its way.

"Interpreted correctly," writes Rothstein, the analyses in the Gates' report "undermine rather than validate value-added-based approaches to teacher evaluation." Jesse Rothstein is not just any economist; not only has he studied VAA in the past, but he served as senior economist for President Obama's Council of Economic Advisors and chief economist at the U.S. Department of Labor. So he is well-equipped to take on the entire stable of Gates-funded economists, mano a mano.

Thompson warned Goldhaber that economists should think seriously about the damage their methods will wreak on teachers and children and schools, especially low-income schools, where it is harder to get big test-score gains. In one of Thompson's perceptive comments, he wrote, "If teachers have a one-sixth, one-fifth, or one-third chance per year of being wrongly indicted as ineffective, none will ever have any peace of mind.
Effective teachers with self-respect will flee those schools for lower-poverty schools."

I was reminded of a comment by Rutgers economist Bruce Baker, who asked whether you would buy a car if the salesman assured you that it would explode only once every five times you turned the ignition key.

If we step back a bit, Deborah, don't you think there is a certain kind of madness in thinking that economists who never set foot in a classroom can create a statistical measure to tell us how best to educate children? It seems some will never be satisfied until they have a technical process to override the judgments of those who work in schools and are in daily contact with teachers and children. I don't know of any other nation in the world that is so devoted to this effort to turn education into a statistical problem that can be solved by a computer. It is not likely to end well.

- Surely it is madness when we are
depriving children of a childhood - no
art, music, recess, P.E,, etc.

- Surely it is madness when we ignore
our own common sense, i.e., of course
poverty matters, of course teachers
aren't perfect, and....of course it's
NOT OK for people who know zilch about
children and learning to make the
big policy decisions.

I'm thinking the root word in madness is 'mad' - if everything a big anger issue?

Has anyone ever talked about how Arne Duncan put no thought into the way he set up charter schools in Chicago? Like the fact that some charters get 95% funding and some only 75% funding. Or that some charters get wrap arounds services funded by CPS, and others don't?

As usual, Ravitch speaks with the force of reason in an increasingly irrational time period in public education. Isn't it ultimately ironic that teachers rated high on their ability to engage students in critical thinking and reasoning scored low on the "Gates" teacher evaluation system? Economists have shown the American public the danger of "pr spin" as pr spin can make anything seem very unlike the reality... think Enron... think mortgage crisis... think Bernie Madoff... Ravitch points out what should be very obvious to all... economists are not experts in childhood development and learning and they should more importantly get their own affairs in order (yes they have made a disaster of our national and world economy)!

It starts with Carolyn Hoxby's preposterous analysis of school choice. Her dissertation merits discussion for its influence on advocates of school choice and vouchers despite its ludicrous assumptions.

I would encourage readers of this blog to click on the above hyperlink, "I signed onto a statement" as it is a great eye-opener that explains the inherent problems of value added assessments of teachers.

We encourage users to analyze, comment on and even challenge washingtonpost.com's articles, blogs, reviews and multimedia features.

User reviews and comments that include profanity or personal attacks or other inappropriate comments or material will be removed from the site. Additionally, entries that are unsigned or contain "signatures" by someone other than the actual author will be removed. Finally, we will take steps to block users who violate any of our posting standards, terms of use or privacy policies or any other policies governing this site. Please review the full rules governing commentaries and discussions.