Limning Minnesota: Reviewing Ripley's portrait!

How much of her portrait is false: For starters, let’s undertake a bit of review:

In her widely-praised book, The Smartest Kids in the World, Amanda Ripley paints a flattering portrait of Minnesota’s public schools.

“Eric” is one of three exchange students Ripley followed to foreign lands. In this passage, she suggests that Eric was lucky to come from Minnesota, given the state’s success with the teaching of math, the result of a great improvement:

RIPLEY (page 72): Of the three American students I followed, Eric was the only one who did not loathe math. Coincidence or not, Eric’s home state of Minnesota was one of only two states that came close to achieving world-class math performance. Roughly speaking, Minnesota ranked below just a dozen other countries (including Canada, Korea and Finland) in math proficiency; only Massachusetts did better in the United States.

When Eric arrived in Korea, he had a solid math background. There were lots of reasons for this: One might have been that his timing was good. Had he been born earlier, things might have turned out different.

In 1995, Minnesota fourth graders placed below average for the United States on an international math test. Despite being a mostly white, middle-class state, Minnesota was not doing well in math. When Eric started kindergarten two years later, however, the state had smarter and more focused math standards. When he was eleven, Minnesota updated those standards again, with an eye toward international benchmarks. By the time he went to high school, his peers were scoring well above average for the United States and much of the world. In 2007, Minnesota elementary students rocked a major international math test, performing at about the same level as kids in Japan.

What was Minnesota doing that other states were not? The answer was not mystical. Minnesota had started with a relatively strong education system. Then they’d made a few pragmatic changes, the kind of common sense repairs you would make if you believed math was really, truly important—and that all kids were capable of learning it.

Nothing we will say this week is meant to be a criticism of Minnesota’s schools.

That said, Ripley paints a very flattering portrait of The Land of Lakes. (Insert pun here. Recommended: “Butters it up.”) The state comes close to achieving world-class status in math, she says. Only Massachusetts does better—and Minnesota’s improvement has been quite impressive:

In 1995, Minnesota’s fourth-graders were below average for the U.S., we are (foolishly) told. But only twelve years later, “Minnesota elementary students rocked a major international test, performing at about the same level as kids in Japan.”

As we will see in the next few days, that passage is larded with embellishments and misunderstandings. We don’t mean that as a criticism of Minnesota’s schools. But Ripley’s book is full of embellished tales, and this is just one more.

In that passage, Ripley is describing Minnesota’s performance on the “major international test” known as the TIMSS, although she never names the TIMSS and ignores it almost everywhere else in the book. (Full name: The Trends in International Mathematics and Science Study.) In this earlier passage, she is referring to the state’s performance on a different international test, the PISA (The Program for International Student Assessment):

RIPLEY (page 47): Before he’d even left the United States Eric was, in some ways, living in a different country than Kim in Oklahoma. Minnesota was one of the very few states that ranked among the top twenty nations in the world in education outcomes. Minnesota did not make it into the top tier with Finland or Korea, but in math, the state’s teenagers performed about as well as teenagers in Australia and Germany.

Even by those standards, Eric had attended a particularly high-powered high school. Newsweek regularly ranked Minnetonka High School among the top high schools in America. The place had four gymnasiums and a hockey rink and looked more like a small college than a high school.

Eric had opted to join the International Baccalaureate Diploma Programme, an intense track within the school that was benchmarked to international standards. He had several teachers who were legendary in Minnetonka...On paper, anyway, Eric was going from one of the smartest states in the United States to one of the smartest countries in the world.

Again, Ripley paints a flattering portrait of Minnesota. Or does she? Let’s fill in some of the background:

On the 2011 TIMSS, the United States outscored both Australia and Germany by statistically significant margins in Grade 4 math. The U.S. outscored Australia in Grade 8 math, though only by a small margin; Germany didn’t take part in the Grade 8 testing.

What about Minnesota? On the Grade 8 level, Minnesota took part in the 2011 TIMSS as an independent entity. Result?

Minnesota outscored Australia by a wide margin on both the math and science tests. But so what? On page 47, Ripley is restricting herself to PISA results, although her readers don’t know that. On that basis, she only says that Minnesota teens perform “about as well as teenagers in Australia and Germany.”

You can’t exactly call that false. But basic information is being withheld, as is the case all through this scattershot book.

In Grade 8 math, Minnesota outscored Australia by a wide margin on the 2011 TIMSS! The United States outscored Australia and Germany, in the manner described, on that same “major test.” But alas! These are the types of facts which keep disappearing from Ripley’s book, due to her general avoidance of TIMSS scores.

In truth, Ripley’s book is a chaotic mess in its use of international test scores. She’s very good at human interest writing, appallingly weak when it comes to the most basic uses of test scores.

At any rate, Ripley consistently portrays Minnesota as one of “the smartest states.” (Forgive the childish language. It’s meant to draw you in, slow learner that you are.) Only Massachusetts ranks with Minnesota in math, she says on page 72.

That’s a very shaky claim. In truth, we’d have to say it’s just false. In the next few days, we’ll flesh out the fuller picture.

Simply put, Massachusetts isn’t the only state which ranks with Minnesota in the teaching of math. But Ripley doesn’t seem to know squat about the ways to analyze test scores.

What else is new? Because we live in a post-journalistic culture, a highly amateurish book is being lavishly praised. This is very much the way our “journalistic” world works.

7 comments:

I'd say that to call the book "scattershot" is being extremely kind. The book is a blatant exercise in shameless bullshit.

What about the chart on page 3 of Ripley's book that shows Italy and Norway plunging to the bottom in test scores? The chart is from an unpublished (needless to say, not peer reviewed) book by soi-disant "educational economists" Hanushek and Woessman, based on an measurement undisclosed "yardstick" invented by those authors, who are supported by a right-wing think tank.

Ripley says in these same early pages that throughout her book she is going to rely mainly on the PISA test, though she admits there are other kinds of international tests (ones less favorable to her argument, doubtless). Why then doesn't she show a chart displaying the results of the PISA tests (available to anyone on wikipedia and elsewhere)?

On the PISA tests that Ripley's claims her book is based on, Italy raised its scores in math by 16 points from 2003 to 2009. Norway also raised its math scores three points in the same period, lowering its reading score by two points to 503 (the USA is 500, presumably average, if 500 is average). The PISA scores for Norway and Italy by no means show the kind of dramatic plunge displayed for them on Hanushek's chart. If Ripley is really basing the argument of her book on the PISA tests, why not display the actual results of the PISA test????

Ripley explains only that Hanushek and Woessman have devised their own "yardstick". She does not share with readers how their "yardstick" is derived. Nor does she comment on the enormous discrepancy between Hanushek's chart and the actual PISA results, remarking only that the economists' chart shows that international tests can rise and fall in a short period. Seriously, what is the point of that information for her argument?

This is so dishonest it makes me wonder if she really wrote her own book or had the propagandists at the New America foundation whom she thank in her introduction touch it up for her. Ripley is not manifestly not stupid. presumably she also had editors. What gives? It's an outright con job. -E