Education Next is a journal of opinion and research about education policy.

The First Negative Effects of Vouchers and the Predictable Misinterpretation

An NBER Working Paper made waves last week because it found large negative effects. It is the first study to find large negative effects of vouchers on achievement. It’s well done. I believe it.

The question is, why? Why are the effects so negative when prior studies have found either no effect or positive effects? Good question. Unfortunately, we know much less about reasons than some have suggested.

The libertarian Cato Institute chimed in quickly: “Although not conclusive, there is considerable evidence that the problem stemmed from poor program design. Regulations intended to guarantee quality might well have had the opposite effect.” Others on the pro-voucher side had similar responses–not surprising for those who want the government as uninvolved as possible.

These interpretations are quite premature, however. The only real “evidence” consists of this single case of Louisiana with heavier regulation and negative effects. I would make the usual warning that correlation isn’t causation, except calling this a “correlation” is a bit generous when the pattern is identified from a single example. Not only do we need more evidence on regulation effects, but we also have to consider evidence on alternative explanations.

There are two other obvious explanations for why the results are worse here in Louisiana. First, Louisiana has especially aggressive accountability for the public schools. In New Orleans, regulation and accountability seem to have contributed to large positive effects on achievement. Why does this matter? Because the voucher studies compare students who won a voucher to those who did not–and those not receiving a voucher very likely ended up in the new and improved public/charter system. Families have to be low-income to receive a voucher in Louisiana so public schools represent the main plausible alternative.

Put differently, even if the quality of voucher-participating private schools was identical in every city, they would all show different effects depending on the academic effectiveness of the publicly funded alternatives. This interpretation creates a tension within the reform community because it potentially pits public school reforms like charters and school closure against reforms like vouchers. The better one looks, the worse the other looks.

A second possibility is that the curriculum in the private schools in Louisiana simply doesn’t align with the state standards captured on the tests. Even with the “heavy regulation” on private schools that voucher supporters decry, the public schools have a much larger incentive to align their instruction than the private schools accepting vouchers. What makes Louisiana different from other cities is not just that they are regulating more heavily, but that voucher recipients take the state test, not some off-the-shelf norm-referenced test that is more in line with the ones that private schools normally use. Weaker alignment between the curriculum and the test in Louisiana would also tend to make the results look worse.

The reform community sure has itself tied up in knots on this one. Even among those who support choice and non-governmental providers, there is disagreement over the rules of the road. Unfortunately, at this point, the Louisiana case tells us little about regulation effects. We can only speculate on the reasons why the Louisiana voucher effects have been so negative, but the fact that they are negative is important by itself.