The findings, released last week, emerged from an analysis of data
that yielded a different conclusion a little more than a year ago.

In February 2002, Paul E. Peterson, a professor of government at
Harvard University, unveiled the results of a study he had conducted
with Mathematica Policy Research Inc. The study showed that
African-American students who had won privately financed tuition
vouchers in a 1997 lottery scored 5.5 national percentile points higher
on standardized tests three years later than did black peers who had
sought but did not receive the vouchers. The researchers called the
difference a "statistically significant positive impact." ("Voucher Plans' Test Data Yield
Puzzling Trends," Feb. 27, 2002.)

Intrigued by the Peterson study's finding that black students
appeared to benefit from the vouchers while Hispanic students did not,
Alan B. Krueger, a professor of economics and public policy at
Princeton, sought to examine the data. Mathematica, a private,
Princeton, N.J.-based group, made it available.

At an April 1 news conference here, Mr. Krueger argued that design
features of the original study had produced misleading conclusions.

"This research has really been blown out of proportion," said Mr.
Krueger, accompanied by his Princeton co-researcher, Pei Zhou, and two
of the Mathematica researchers who had worked with Mr. Peterson.

"For the most representative sample of black elementary school
students," he said, "offering a voucher had no statistically
discernible impact on achievement scores in the New York City
experiment."

Standing Firm

Mr. Peterson, who was invited to the news conference but did not
attend, said he is working on a paper to respond to the technical
questions raised by Mr. Krueger's analysis. He said he stands by his
original findings.

"Our original estimates, at least as far as we know at this point, are
the best available estimates of the impact of the voucher program," Mr.
Peterson said.

Mr. Krueger, in detailing the aspects of the earlier study that he
contends produced skewed results, pointed to the exclusion of large
numbers of children from the sample. While 2,666 needy K-4 pupils were
placed either in a group that received vouchers or a control group that
applied for but did not get them, more than 500 were excluded from the
study because they did not take the baseline standardized test.

Bringing those children back into the study sample increased its
size by 44 percent, Mr. Krueger said.

The definition of race proved pivotal as well, he said. The original
study inferred a child's race or ethnicity by asking the female parent
or guardian to choose one from a list of racial and ethnic groups to
describe herself. That method excluded many children, including those
of mixed race and those whose fathers might belong to a racial or
ethnic minority.

Adding those children back in increased the sample size by another
10 percent, he said. With those additions, and more revisions made to
control for other demographic factors, Mr. Krueger and his colleagues
found that black children who had received vouchers scored only 1.44
percentile points higher on tests three years later than did their
black peers who did not receive vouchers.

Mathematica researcher David E. Myers said the re-evaluation had
been "collegial," and a model of how social scientists should share
their findings and allow the data to be subjected to multiple
examinations. But he lamented what he sees as the way some academics
and reporters magnified the differential between black voucher
recipients and nonrecipients. They overlooked what he considers the
most significant finding, he said, which is unchanged by Mr. Krueger's
analysis: In the overall group of students studied, obtaining vouchers
provided no academic benefit.

Vol. 22, Issue 30, Page 13

Published in Print: April 9, 2003, as Study: No Academic Gains From Vouchers for Black Students

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.