Is progress in Florida schools exaggerated?

By
Valerie Strauss

Florida’s public school system is frequently held up as an exemplary example of how specific school reforms -- vouchers, charter schools, high-stakes standardized tests, etc. -- can improve student achievement.

Former Florida Gov. Jeb Bush talks frequently about the great successes he had in his eight years in office (his term ended in 2007) and afterward through his Foundation for Excellence in Education, while officials in other states compare their own progress with that of Florida’s, and think tanks write papers about Bush’s accomplishments.

"One state, Florida, has demonstrated that meaningful academic improvement—for students of all races and economic backgrounds—is possible. In 1999, Florida enacted far-reaching K–12 education reform that includes public and private school choice, charter schools, virtual education, performance-based pay for teachers, grading of schools and districts, annual tests, curbing social promotion, and alternative teacher certification. As a result of parental choice, higher standards, accountability, and flexibility, Florida’s Hispanic students are now outperforming or tied with the overall average for all students in 31 states. It is vital that national and state policymakers take the lessons of Florida’s success to heart. The future of millions of American children depends on it.

The paper discusses fourth grade standardized test scores in reading -- because, it says, they “are an important focus of education reform because early childhood literacy is the gateway to all other learning.”

Florida students, it says, have “demonstrated the strongest gains” on the National Assessment of Educational Progress in the nation since 2003. And, it says, “After a decade of K–12 education reform, Florida’s Hispanic and black students have outscored averages for many other states."

And because of Florida’s success, other states should adopt the same kinds of reforms.

That all sounds good, but, it turns out, it’s not quite all that it seems.

A new review of the report, by Madhabi Chatterji of Teachers College at Columbia University, questioning its methodology and citing flaws that undermine its key conclusions.

For one thing, the Heritage paper fails to discuss or consider the impact of the voter-approved class-size reduction policy launched across the state in 2003-04. It also does not discuss the influence of “fundamental policy changes on test score averages and racial achievement gaps in grade 3-4 students.”

“In particular, Florida instituted a grade retention policy from 2002 that resulted in 14-23% of largely Black and Hispanic third-grade students being held back in grade 3 if they performed poorly on the state reading test. This policy of screening out the weakest readers, along with the presence of unknown numbers of older grade repeaters in the grade 4 samples, changes the composition of the students tested in grade 4 and invalidates comparisons concerning student performance as a whole as well as results concerning ethnic group achievement gaps."

Another flaw noted was “the decision to look only at grade 4 NAEP Reading scores and the resulting inflated conclusions.

“The evidence on Florida’s NAEP achievement trends and gaps is mixed when other grade levels and subject areas are examined between 2002 and 2009,” it says.

There are a lot of other elements to Florida’s education reform story that bare scrutiny. This serves to remind us that things are often far more complicated than they seem.

Florida has some good things happening in its schools but as Diane Ravitch is often quick to point out, whenever high stakes are attached to student tests the risks of gaming the system are increased exponentially.

Why is it that the left-wing Teachers College of Columbia University falsely criticizes the conservative Heritage Foundation's report:

"It also does not discuss the influence of “fundamental policy changes on test score averages and racial achievement gaps in grade 3-4 students.”...“In particular, Florida instituted a grade retention policy from 2002 that resulted in 14-23% of largely Black and Hispanic third-grade students being held back in grade 3 if they performed poorly on the state reading test."

Yes it did!

"Social Promotion Ban. Florida has also curtailed the “social promotion” of students. The reform plan requires students to pass the third-grade reading FCAT before progressing to fourth grade. Despite the nation’s limited progress in raising the scores of minority students overall, Florida has made enormous progress in narrowing racial achievement gaps."

So now I have to ask, why does the left-wing WAPO feature this false allegation? Did Ms. Strauss even bother to read the Heritage Foundation's report before running to Columbia for an attack on legitimate research?

I happen to know Lindsey Burke and know she performs excellent research, double and triple checks her facts, is well-spoken and has integrity. Could it be that Progressives simply don't like the fact that her report demonstrates a "politically unpopular" solution to closing the achievement gap actually works?

First, on a personal note, I agreed to set my youngest son back into the 3rd grade when a teacher advised me he would continue to have problems if not corrected early. It was by far the best move I made. He graduated a year later than might have been, but he graduated with far better grades.

Does it matter that he would have been included in the mix of children if he were tested the second time? NO! Does it matter that he was 10 instead of 9? NO!

One group disregards or at best has apprehensions about the Florida initiative. So what? Given where they were vs. where they are is an improvement (living in Florida I have seen the difference.) Is that an academic crime?

Listen, I would rather have a youngster graduate high school at 20 knowing they get it, than having someone drop out at 16 because they gave up on getting it.

lisamc31 - Personally, I think it's great that kids are held back in order to get up to grade level. That helps them be ready for advanced learning and not get stuck in a cycle in which they are moved ahead without ever having a chance to make progress.

Unless all schools around the country are doing that, it also has the effect of skewing the 4th grade NAEP scores.

Also, did you know that NAEP is given only every two years? That means that kids held back in 3rd grade won't get tested when they're in 4th grade, so they could slip through the system even though their reading still may not be up to par.

Take a look at the fraud being perpetrated in the Florida high schools. Regularly among the lowest AP passing percentages in the country (based on test takers). Of course Fla. touts participation and then computes passing percentages based on the entire class of graduating seniors, whether they tested or not.
For example, look at some of the regular Fla. suspects (like Hillsborough High) on Jay Mathews list of best high schools top 50. Bunch of those schools have loads of kids taking AP tests and almost nobody passing them.

To answer your question, yes, it is possible that Florida is closing the achievement gap in grade 4. However, it is also possible that she is wrong in her conclusions for what brought it about. It could simply be a mistake in cause and effect in her research because she is dealing with multiple social variables. Any competent reader of this type of research could tell you that.

I highly suggest you purchase some unbiased glasses that don't leave your brain blind to the obvious when dealing with humans and the possibilities that arise when dealing with them. Reading some mystery stories might help, too. Or perhaps, take a basic graduate class on reading and evaluating this type of social research.

By the way, almost all states are closing the achievement gap in grades 2-4. Where have you been? The big problem comes in after 4th grade where the gap gets bigger and in some cases triples in middle school and high school. And guess what? Florida is no better than many other states in grades 5-11.

Anything that comes from a think tank, whether on the right or left is written to make the facts fit the agenda. By its very nature, it is propaganda and should be examined with jaundiced eye for its selective nature. To say that based on fourth grade scores that all of the reform measures are causal is a huge leap that defies logic. While it is logical to credit their no social promotion policy for a rise in fourth grade scores, to give credit to alternative pathways, charter schools, performance-based pay schemes or virtual education would be unsupportable by the evidence.

In other words, partisan think tanks are not a source of reliable information upon which to make public policy.

On a completely anecdotal level, however, I always wonder why that over the past 33 years of teaching in New York's public schools that even the better students that come from Florida need to make up courses, and often don't graduate with their age peers.

If you actually talked to teachers in the Florida Schools they would tell you how the system has changed int he last ten plus years. The curriculum has narrowed to focus on whatever is tested in your grade level to the exclusion of things like social studies and science that for many years were relegated in elementary school to an hour a week or less. That is changing for science but only because it is tested and the tests count for school grades. My own middle school science classes are constantly being curtailed in order to fit more and more diagnostic testing into the schedule. We test more in a desperate attempt to pass the high stakes tests. Many of the labs and projects I used to do are either out completely or down to a one period quick run through. What all of us want more of is time to teach but that is declining each year. In the math and scripted reading classrooms it is much worse and conflicting mandates to "fix" our school's math score have its teachers struggling to incorporate 3 different full time programs into a 50 minute period while preparing students for a mandated diagnostic quiz every month that does not address the same topics as the other three programs. The only ray of sunlight I see in the system is the class size which the legislature is working hard to eliminate along with the mandate for it to fund public education. That voter initiative has allowed me to respond better to individual needs and to do more hands on activities when time allows. It is amazing to me that so little information in this article and in the comments here comes from the people who work with the children all day, their teachers, and the people who welcome them home every afternoon, their parents. Instead we have reams of incomprehensible test data and the opinions of "experts", apparently defined as anyone who resides at least 50 miles from a classroom full of children.
And Valerie I do love your article on how to tell how children are doing in school. Excellent!

I get a kick out of buckbuck above telling people that think tanks aren't reliable based upon bias. And Columbia Teacher's college?! No one is more biased, skewed, and abusive of facts than the educational establishment defending their decades-long legacy of failure.

A few of thoughts on this post.

1. The Columbia study touting class size is instructive of bias. Class size reduction leading to improvement is one the most enduring myths of education dogma. It does wonders to employ a lot of people, but the massive investment in reducing class size has done nothing for America's overall standing. It's time to stop throwing good money after bad.

2. Perhaps the best howler from Columbia is positing that holding back kids who don't learn might 'skew the scores' upward. Well DUH! Why promote someone beyond their ability? To be sure, the kids who didn't get promoted based upon ability need to be placed in some re-mediated/accelerated program to get them up to speed. The money for such a program should come right out of budget of the school that failed them.

3. I, for one, wish we could focus on something more than test scores. While content uptake is important, we ought to focus much more on the wishes of the family and the child.

Decades of mandates and increased spending are proven to be an utter failure, mostly because the money is filtered through a greedy, entrenched, and protected industry.

Tests lead us to only look at part of the problem. If people were given a choice as to how their education dollar should be spent, they would pick the option best suited to their needs. Scores would rise, but not dramatically. Socially, however, the system would be much more functional and conducive to learning, broadly, how the world around us works.

We need to stop "reforming" this awful education system, and begin dismantling it. Out with districts, administration, unions, bond dealers and the like. In with 100s, if not 1000s, of independent content providers whose income is derived from meeting customer demands.

We ought not need a study to understand that such as system is light years better than today's system of controlled chaos.