A new report on the academic performance of low-income students receiving Tax Credit Scholarships in Florida finds they are making modestly larger gains in reading and math than their counterparts in public school.

That conclusion from 2009-10 test data is encouraging for those of us who work to provide these learning options, which served 34,550 low-income students statewide last year. But the report, released today and written by respected Northwestern University researcher David Figlio, is also a reminder of the inherent complexities of judging whether these programs work.

Figlio has both a brilliant mind and 13,829 test scores with which to work, and yet his report is filled with qualifiers and provisos and cautionary notes. That’s largely because the scholarship program is so different from the typical public education option. In this case, students are attending more than 1,000 private schools where, on average, four of every five students pay their own tuition. The average scholarship enrollment in each school, for 2009-10, was only 28 students.

That kind of school profile tends to serve as an asset to the economically disadvantaged students, but not necessarily for the standard approach to academic oversight. Since these are mostly private-market schools, the state won’t allow them to administer the state test, known as the Florida Comprehensive Assessment Test (FCAT). But the law does appropriately require every scholarship student to take a nationally norm-referenced approved by the state Department of Education, and most students take the well-regarded Stanford Achievement Test.

These tests do allow Figlio to make direct national comparisons, so we know without qualification that the typical scholarship student scored at the 45th percentile in reading and the 46th percentile in math. We also know that their year-to-year gain from 2008-09 to 2009-10 was the same as students of all income levels nationally, which is a solid piece of academic evidence

Where things get more muddled is in trying to compare to low-income students in Florida public schools. As odd as this may sound, the two groups are substantially different. And they are different in ways that tend to be counterintuitive.

Phi Delta Kappan today released its annual poll on public school attitudes, and it found mixed results for the support of school choice. The poll found increased support for charter schools and choice generally, but Kappan found that only one in three Americans likes vouchers.

That’s little surprise, given the way the voucher question was asked:

Do you favor or oppose allowing students and parents to choose a private school to attend at public expense?

Earlier this month, Education Next and the Program on Education Policy and Governance at Harvard University released the results from a similar poll and found record levels of support for vouchers. That disparity might be attributed to the way Education Next-PEPG addressed the issue.

When it came to private options, the poll sought answers through several different questions. It first randomly assigned respondents a “voucher-friendly” question:

A proposal has been made that would give families with children in public schools a wider choice, by allowing them to enroll their children in private schools instead, with government helping to pay the tuition. Would you favor or oppose this proposal?

It then randomly assigned a “voucher-unfriendly” question:

A proposal has been made that would use government funds to help pay the tuition of low-income students whose families would like them to attend private schools. Would you favor or oppose this proposal?

Not surprisingly, more people say they like vouchers if asked the friendly question (47 percent) than if they were asked the unfriendly question (39 percent). Support also increases across the board if the private option takes the form of a tax credit scholarship. Additionally, Education Next breaks down support by race and shows that black and Hispanic groups overwhelmingly support private options compared with affluent respondents or with teachers.

This is not meant to discount the sweep and significance of the Kappan poll. I flirt with these comparisons only with the hope that headline writers heed these subtleties before we read that “Charters are in, vouchers are out.”

The authors of the latest Education Next-PEPG Survey highlight the growing disconnect between the general public, the affluent and teachers when it comes to sweeping public policies in education. But, just as notably, the results show a wide range of attitudes between the affluent, Hispanics and African Americans when it comes to school choice.

Vouchers have gained more support nationally since the 2010 survey, but support slips when the results are broken down by the affluent and by teachers. In some cases, the difference is stark among minority groups and the affluent, but those differences disappear when the policies (and the questions) change.

Depending on how the question was asked, as much as 60 percent of Hispanic respondents and 53 percent of African Americans supported vouchers compared to 47 percent of affluent respondents.

However, when it comes to individual or corporate tax credit scholarships, support among the affluent increases to 57 percent, which is the same result among African Americans and closer to that of Hispanics, a group that showed no difference in support among tax credits or vouchers.

Adam Schaeffer at the Cato Institute has more on the differences in support of vouchers and tax credits here.

UPDATE: A team of university researchers is releasing data showing more comprehensive results on the performance of students in the Milwaukee Parental Choice Program than the state of Wisconsin has shown, according to a story in today’s Milwaukee Journal-Sentinel. The team, which includes professors John F. Witte of the University of Wisconsin and Patrick J. Wolf of the University of Arkansas, have tracked the performance of a sampling of children in the choice program over three years and found that the students performed about the same as their peers in Milwaukee Public Schools, not worse. The day before, the Wisconsin Department of Public Instruction released data showing that half the students at either setting read at grade level, but that district students far outperformed choice students in math. The university team also found that a sampling of ninth-graders in the voucher program had slightly higher rates of graduation and enrollment at a four-year college than a matched sampling of students in the school district.

The results of Milwaukee’s first comparative assessment of students in the Parental Choice Program and those of their peers in the school district have uncorked the kind of responses one might expect from an education policy that has divided the community for more than 20 years. But that does more to highlight the political strains of the voucher program than it does to explain the performance of its 21,000 students.

This is not to dismiss the Wisconsin Department of Public Instruction’s data, which showed that the low-income students in the choice program performed similiarly to their traditional public school peers on free or reduced-price lunch in some ways, and worse in others. About half the students in either setting are reading at grade level. But only 34.4 percent of choice students scored proficiently in math, compared to 43.9 percent among low-income pupils at Milwaukee Public Schools.

That’s certainly not good news for the choice program, but it’s hardly the occasion to tell the low-income parents who’ve chosen to participate that they’ve been “bamboozled,” as one Democratic representative told the Wisconsin State Journal. As University of Wisconsin political science professor John Witte noted, “in order to study achievement growth and gain, you have to study individual students over time.” Witte has been among the most clear-eyed and careful scholars to study the academic impact of school vouchers generally and the Milwaukee program specifically, and his careful response to yesterday’s news should better inform the state’s own superintendent of instruction. Shamefully, state Superintendent Tony Evers distributed a news release statewide showcasing that Milwaukee public schools do it better.

Such a move from Wisconsin’s top educator does nothing to advance the debate over how best to educate our most disadvantaged children in the 21st century. We have a growing array of educational alternatives from which to choose in our public education systems and we should be careful to avoid singling out one option as better than another. Milwaukee’s program was created in 1990 at the urging of a Democratic representative who wanted to empower her low-income and mostly minority constituency with the same ability to choose a private or even faith-based alternative that wealthier families had long enjoyed.

This response may seem to avoid the reality of the data. I don’t argue that test scores are insignificant, but just as in traditional schools, they are best judged over time. Florida’s tax credit scholarship for low-income students suffered the same criticism two years ago. Northwestern University professor David Figlio examined the performance on the Stanford Achievement Test of students in the scholarship program, as commissioned by the state, and found they made the same gains as students of all income levels nationally. The same achievement was not good enough for critics, but Figlio later cautioned against a rush to judgment. “I feel we need to have stronger causal evidence on the relative effectiveness of the program,” he told the St. Petersburg Times.

All schools need to be held accountable for learning, and Milwaukee’s record of reaching low-income students through either traditional programs or choice leaves considerable room for improvement. But after 20 years, Milwaukee’s public school system should have learned to co-exist with schools like St. Thomas Aquinas Academy or Yeshiva Elementary, which can rightfully be called “public” by any definition. Instead of thumbing his nose, Superintendent Evers should work to find common ground to ensure the poorest and lowest-achieving among us enjoy every opportunity that meets their needs.

In his latest Time.com column, Andy Rotherham provides a fair-minded appraisal of the school voucher debate as he attempts to disspell the common myths that are tossed around like rhetorical hand grenades. Vouchers don’t drain money from traditional public schools, Rotherham argues, nor do they skim the best students. On the flip side, he says, we need more evidence to support the contention from some that vouchers lead to higher academic achievement and that the resulting competition for students leads to greater results overall for public schools (although on this note, Rotherham does reference the results from a recent study of the Florida Tax Credit Scholarship which found that the competitive effect boosted the academic performance of public schools faced with the threat of losing students).

Notably, Rotherham concludes his column with a statement that arguably should guide the debate over school choice, but too often does not:

Parents should worry a lot less about the legal status of a particular school than whether it’s the right school for their child. A good fit depends on a host of factors including a strong academic program, successful outcomes, a clear curriculum, areas of emphasis like arts or technology, and even lifestyle factors such as limiting time spent in transit or a year-round schedule.

Today’s Philadelphia Inquirer devotes considerable attention to the impact school vouchers have on public schools. At a time when opponents to publicly funded private learning options are lobbing rhetorical hand grenades in several states, particularly in Pennsylvania and New Jersey, Inquirer reporter Adrienne Lu offers this fair-minded assessment:

While studies are relatively scarce, the early opinion among researchers appears to be that vouchers have done little, if any, harm to student achievement in public schools and in some cases have spurred small improvements on standardized-exam scores in public schools.

As evidence, Lu cites Northwestern University researcher David Figlio, who recently found that the Florida Tax Credit Scholarship boosted the academic performance of the public schools faced with the threat of losing students to the program. Figlio and co-researcher Cassandra Hart had highlighted that, no matter what measure they used (the closer private schools are to a public school, the density of private schools within five-miles of a public school, etc.) the effect was generally the same:

Although these effects are relatively small, they consistently indicate a positive relationship between private school competition and student-performance in the public schools, even before any students leave for the private sector. That is, these results provide evidence that public schools responded to the increased threat of losing students to the private schools.

In an interview with the Inquirer, Figlio rightly cautioned against looking at vouchers as “the magical pill that’s going to turn the U.S. into Finland,” but he made clear that, for any state considering a voucher program, “there’s very little to be afraid of.”

Key to improving public education is aligning our practice with what scientists have discovered about human motivation. Daniel Pink, in his 2009 book, Drive, is the latest author to summarize these scientific findings and discuss their implications for enhancing public education.

People are motivated, in part, by what social scientists call “intrinsic motivation.” Intrinsic motivation refers to drives beyond basic survival needs, and Pink identifies three he says should guide teaching and learning: autonomy, mastery and purpose.

People have a natural desire to be autonomous and self-directed. Teachers and students who feel a greater sense of control over their teaching and learning, respectively, experience greater success than their peers who feel less control. Researchers have also found that students who attribute academic performance to hard work, a variable they control, are more successful than students who attribute academic performance to innate intelligence, a variable they cannot control.

This need to be self-directed is one reason school choice is so essential to school improvement. Teachers, students and parents are more motivated and satisfied when they can choose their schools.

The event will be held from 10 a.m. to 3 p.m. and, according to Education Week, “will investigate the impact of the recession, federal stimulus, and broader economic conditions on the nation’s schools.”