Diane Ravitch, lamenting a recent column by Times editorialist Bill Keller, who lazily blamed widespread problems with education performance on university teacher preparation programs (without mustering a shred of evidence to support his claim), concluded, “It would be wonderful if the New York Times elevated someone to the op-ed page as a columnist who actually knew something about education.”

Staying true to form last week was Times opinionator Nicholas Kritsof. Prompted by the latest results of the National Assessment of Education Progress, aka “the Nation’s Report Card,” Kritsof observed on Twitter, “Latest NAEP school test scores suggest that school reform helps. Big improvements in DC & Tennessee, both centers of reform.”

Since 1990, the “Main NAEP,” given every other year in grades 4 and 8 to measure national and state achievement in reading and math, has led to all sorts of overblown claims. This year’s results have been subjected to the same tendencies – despite the fact that the results were described as “stubbornly mixed” by reliable news outlets, with stagnation in 4th grade reading and math and slight gains in 8th grade reading only.

Unfortunately, delusions suffered by Kristof and his cohort are not limited to pages of The New York Times.

Cue Up The Conservative Messaging Machine

It’s perhaps understandable why Kristof would make the claims he does about the NAEP results, given the level of propaganda pumped into the airwaves by those who claim to be “reforming” education.

Hard at work in the messaging mill every day are a plethora of highly paid operatives in conservative “think tanks” whose missions are to impose their policy prescriptions on public schools.

These outlets were quick to jump on just the sort of statistic Krstof did. Because some states that have implemented the favored policy reforms had gains in their NAEP scores that outpaced the national average, these “analysts” made the case that “reforms” – such as, using high stakes test scores to rate teachers and schools and increasing the numbers of unregulated, privately operated charter schools – are “working.”

To amplify this message, Michelle Rhee went on camera at MSNBC to claim score gains produced by the District of Columbia, the school system she once led, and Tennessee, a school system she has had considerable influence in, were vindication of very controversial policies that led to her ouster from the D.C. system.

Such claims are truly silly.

As sociologist and professor Aaron Pallas at the Hechinger Institute pointed out, the difference between the NAEP scores from this year and those in the previous year can’t be viewed as “indicator[s] of growth or decline.” To do so would prove only how little you understand about statistics.

To ascertain any real growth or decline in students’ academic achievement, NAEP would need to measure the very same individual students it did in previous years. “But NAEP does not measure the same individual at two points in time. Instead, it measures different individuals at each assessment,” Pallas explained (emphasis original).

The 4th-graders and 8th-graders that NAEP assessed for this year in Tennessee and the District of Columbia aren’t the same ones that were assessed in previous years. Their demographics are different. Their experiences are different. “These changes in demography could account for differences in the average performance.”

Still No Proof Rhee-form Works

Claims about D.C.’s NAEP scores as proof of the effectiveness of reform policies implemented by Rhee are a particularly big stretch.

Writing at the site Public School Shakedown, edu-blogger Guy Brandenberg looked deeper into the data and concluded that the District’s NAEP gains “are a continuation of a trend that has been going on since about 2000 … You will have to strain your imagination to see any huge differences between the trends pre-Rhee and post-Rhee.”

Brandenburg, and others, point out that D.C.’s NAEP gains this year shouldn’t obscure the fact that the District still scores “dead last in the nation.”

Statistical sense aside, the editorial board of The Washington Post nevertheless fell in line with reform propagandists to trumpet, “School reform in the District is working.”

This prompted Bob Somerby, a former teacher who now blogs at The Daily Howler, to note, “Score gains were actually larger before 2007″ when Rhee’s tenure began.

“The board should stop playing cheerleader in this arena,” Somerby admonished, “and start behaving like journalists.”

Reform “Works,” Whatever The Results

Yes, behaving like journalists would be good. So would behaving like public policy leaders – something that U.S. Secretary of Education Arne Duncan seemed inept at when presented with NAEP results.

Duncan was first out the gate with false claims about what NAEP results proved in terms of education policy.

Quoted in Education Week, Duncan pointed out that that two states, Tennessee and Hawaii, and the District of Columbia which made strong gains in NAEP scores were also early adopters of his pet policy, Race to the Top.

“Tennessee, D.C. and Hawaii have done some really tough, hard work, and it’s showing some pretty remarkable dividends,” he boasted.

Education Week reporter Catherine Gewertz also noted, “The secretary also linked adoption of the common core – expectations his administration has trumpeted from coast to coast – to NAEP gains in eight states.”

Isn’t it convenient how Duncan seems to see in NAEP results confirmations of the biases he had before results were ever announced?

That tendency toward confirmation bias is the norm in regards to NAEP. Education Week’s veteran reporter Stephen Sawchuk warned of just this sort of thing before the NAEP scores were released.

NAEP results, Sawchuk explained, “are frequently pressed into service to bolster claims about the effect that policies [have had].” But, “while those assertions are compelling, provocative, and possibly even correct, they are also mostly speculative.”

The Huffington Post’s education reporter Joy Resmovits, noted a similar problem with NAEP commentary, quoting education researcher Mat Di Carlo, who said NAEP “is a measure of student performance, not school or policy effectiveness, and this valuable information is too often lost in a barrage of advocacy and unwarranted causal inference.”

Di Carlo has contended for years that NAEP results are used improperly to bolster arguments about policy effectiveness. “People on all ‘sides’ will interpret the results favorably no matter how they turn out.”

There may be some validity to Duncan’s claims that states that have implemented his policy preferences for teacher evaluations, charter school proliferation, and Common Core – all essential elements in Race to the Top – are producing gains in achievement. But for every state Duncan can identify as a “top performer” on the NAEP assessment, one can point to at least one state or more, as Rutgers University professor Bruce Baker did, that has excelled in NAEP score gains over the years when reform measures were not being implemented.

Does this mean that results on the NAEP are meaningless measures? Of course not.

NAEP. What Is It Good For?

One of the most significant revelations from NAEP is the continued disparity in academic achievement among the nation’s children based on their ethnicity and income level.

Results from NAEP have alerted policy makers and journalists to the “achievement gap” in America long before policy ideas like No Child Left Behind, which has too often been erroneously credited with revealing those gaps, came into being.

What this recent NAEP assessment revealed was very little if any progress on closing these gaps. As Huffington Post’s Resmovits reported, some black-white score gaps have narrowed since 1990, and same for some Hispanic-white gaps. But since 2011, score gaps haven’t narrowed at all.

In its analysis, the Education Trust, an organization devoted to decreasing education disparities, cherry-picked a few states, which exhibited some progress in “gap closing” on NAEP results, but noted that D.C., who reformers identified as a star, actually saw its achievement gap widen.

States that the analysis cited for their “improvement” are truly a mixed bag whose policies can’t be easily generalized. And the analysis confined its scope to the past decade alone whose record of advancement pales in comparison to the gains African American and Hispanic students made on NAEP national assessments in years prior to No Child Left Behind.

“Lots of work remains,” the Education Trust concluded. But work on what?

Reform Harder

Among the education reform community, results from the NAEP assessment, despite any clear signs of significant progress in the nation’s education achievement, have led to the resounding conclusion that there is a deep and urgent need for – you guessed it – more reform.

What else would they have concluded? Had NAEP scores increased or decreased, either way, there would be justification in the reform movement to demand, “We must continue the reform.”

In the meantime, the urgent matter NAEP results really do reveal – that deep, longstanding racial and economic inequities still characterize America’s system of education – get barely any purchase at all.

It’s understandable that reform proponents like Michelle Rhee would make huge overreaches using NAEP results to advance their policy demands. Education reform, after all, is a big business, and the NAEP results appear to provide reform merchants with something to spin for marketing their cause.

But for journalists and policy leaders to parrot these specious claims is inexcusable. The American public deserves better.

Share this:

8 Responses

Dolores Lucerosays:

As a substitute teacher, I have witnessed such deplorable behavior in the class room that I would hate to have a child of mine in this environment. it is not fair for the students who really want to learn to be exposed to such deviant behavior. If the student is fortunate enough to be in Advanced Placement classes in Middle and High School, or in Gifted and Talented in Elementary School matters, are much better and learning and achievement can be accomplished. Something must be done regarding student behavior in the regular Academic classes.

teacher with a brainsays:

Sort of an aside, though related. I was checking up on one of the charter schools featured in “Waiting for Superman” yesterday. Not only does this school have an average SAT score below both the state average and the district (in which it is located) average, it has a much lower poverty rate and English Language Learner rate than the public school where I teach and it is false lauded as a miracle school. Their API is a little higher than at my public high school of 3600, but again, we are over 1/2 low income and they are 20% low income. Test scores do not seem to support that this school is excelling, other than pushing college and claiming that they have a 100% acceptance rate to a 4 year university. They operate a 4-step application process that creates several barriers to applying as well. Class size is no greater than 25, while in my school it is 33-38 and we must educate everyone. I am not very impressed.

BAsays:

Michelle Rhee has already been fired from the D.C. School District. Her methods do not work. What happens is that public schools across the country MUST try to teach students of all races, creeds, philosophies, backgrounds, varying levels of the understanding of how to speak, read, and write the English language, behavioral disorders, learning disabilities, previous educational experiences, and various socio-economic situations. Conservatives and Republicans often look at public schools as factories of education. They are only interested in profits. They are not interested in the individual student. We must listen to Diane Ravitch. It is clear that we have not backed up far enough in terms of how we try to help children achieve. It is absolutely clear that students who live in poverty are much more likely to not achieve as well or as much as their classmates who do not live in poverty. Without going into all of the reasons that poverty still remains such an important issue in the culture of the United States (many good books have been written about this), the ongoing lessening of the number of people in the “middle class,” the continuing aggression against the common working person, the mythology that teachers’ unions keep others from getting jobs and that unions raise taxes, and the failure to truly recognize the importance of each individual person in our society, all lead to lower achievement in public schools. One of the things that happens is that whenever schools try various “band-aids” to try to help students such as free and reduced breakfast and lunch, after school activities, small amounts of counseling, lowering of class sizes, and the like, when those methods don’t give the intended results, the schools are blamed for not doing enough or trying hard enough. The reality is that teachers’ demands are ever-growing. Teachers not only have to teach, they must counsel, they must reteach, they must give standardized tests, they must attend a number of meetings outside of the school day, they must work with parents who don’t know how to deal with their own children, they must deal with administrators who may or may not know anything about teaching or learning, they must deal with in-class “problem” students, they must continue with their own education, they must continue to be certified, and they must not make too much money. Again, we must follow the lead of Diane Ravitch so that public education can succeed. This must also involved getting the President, the Senate, and the House of Representatives to realize the above problems and how to better address them.

Paul Zolbrodsays:

Here’s something I’d like to see investigated. Is there a correlation between the increased use of computer technology and the decline in our education infrastructure? I raise it as an open question, inasmuch as the two developments have accompanied one another. And has that educational decline accelerated with the increased use of data in assesment? A veteran of fifty plus years in the college level classroom and still practicing, here’s something I think I have noticed. With their laptops and notebooks, students have become better at information gathering, not as inclined to think critically, less curious, and disinclined to synthesize.

I think you are exactly right. I have noticed the same thing, mostly in the last 2-3 years, although I have a couple of types of classes and students to compare. Both online and in person (I was creating online classes and software in the 1990′s) – both general studies students and dance students.

I think the distractions of phones and other internet-related connections have led to a lack of focus combined with a skim and skip reading style. Students ask questions (demanding answers) which were already in the online text material and which earlier online students managed to see and comprehend.

Dance students give me an interesting comparison because they are great focusers which is a concentration tool they need to learn and incorporate multiple choreographies and the constant changes. Literally thousands of tiny nuances. They also have the distraction devices but most of their lives are centered around their primary activity, dance. Dance, for all its physicality requires very considerable mental abilities combining musicality, history, physiology and more.

In my in-person class with the dancers, a computer animation class for animating choreography, even those who proclaim (complain?) “computers don’t like me” are able to manage to software and produce great animations of their own choreography. And all this while having to miss a lot of classes because of rehearsals and performances (something the class organization takes into account, aided for material by my online experience while the work itself has to be done at one on-campus computer lab with that software). The physical location of the lab adds to their difficulties in the class and yet they come through in shining colors.

There was always a visible difference between these dance students and the rest of the students but from about the point of the introduction of the iPad this difference has expanded. Although it seems to coincide with iPad I suspect it really is a gradual change starting more with smartphones and especially the iPhone. The shift in distraction may have taken that long to be obvious.

And thinking? Sometimes I wonder whether “thinking” is even the right word. There seems to be more of a match-and-react operation going on. A “plugged in” aspect which I suspect leads to use of packaged “solutions” (I’ve really come to despise the word “solutions” as a meaningless advertising pretense).

Jeff Bryantsays:

Subscribe

Fill in the form below to subscribe to our periodic newsletters.

Street

Street 2

City

State/Province

Zip/Postal Code

Subscription: Education Newsletter

Sign the Education Declaration to Rebuild America

For too long, policymakers have engaged children in a grand experiment, with frequent testing, incentive programs and top-down mandates that promised much but delivered little. Now leading academics, policymakers and educators are demanding an "education spring" embodied in An Education Declaration to Rebuild America.