Offering middle-school math teachers bonuses up to $15,000 did not produce gains in student test scores, Vanderbilt University researchers reported Tuesday in what they said was the first scientifically rigorous test of merit pay.

The results could amount to a cautionary flag about paying teachers for the performance of their students, a reform strategy the Obama administration and many states and school districts have favored despite lukewarm support or outright opposition from teachers’ unions.

The full report is available here (PDF) from Vanderbilt University’s National Center on Performance Incentives (NCPI). NCPI’s press release announcing the report describes the study design as follows:

Following a year of detailed project design by a multi-disciplinary team from Vanderbilt and RAND, all middle-school math teachers in Nashville were invited to volunteer for the experiment. Approximately 70 percent of all middle-school math teachers in Nashville’s public schools stepped forward to participate.

Approximately half of the nearly 300 volunteers were randomly assigned to a “treatment” group, in which they were eligible for bonuses of up to $15,000 per year on the basis of their students’ test-score gains on the Tennessee Comprehensive Assessment Program (TCAP).

The other half were assigned to a “control” group not eligible for these bonuses. Teachers were evaluated based on an historical performance benchmark for MNPS teachers, not on competition with one another. All teachers in the treatment group had the chance to earn bonuses.

The annual bonus amounts were $5,000, $10,000 or $15,000. Over the course of the experiment, POINT paid out more than $1.27 million in bonuses. Overall, 33.6 percent of the original group received bonuses, with the average bonus being approximately $10,000.