Decent PISA Numbers, But Can Mass. Students Really Do The Math?

United States math scores dropped in the 2015 PISA test,, as did the ranking against other nations on that test. (Courtesy/National Center for Educational Statistics)This article is more than 3 years old.

ANALYSIS

Mitchell Chester, the state’s commissioner of elementary and secondary education, was pretty upbeat when we spoke earlier this week.

“If we were a nation, we’d be in a league with the highest-performing nations in the world,” Chester said.

We were speaking about the latest so-called PISA test of 15-year-olds in 72 countries, and he’s right. In the 2015 test results, released Tuesday, Massachusetts made the top 20 in math and the top 10 in science; in reading, it was statistically tied for first place.

Those results have led at least one observer to suggest that Massachusetts is the new Finland – the place to go if you want to see education that works. It’s a lovely idea. But if you dig down a bit, the picture gets a lot more complicated.

Before we dig in, here’s a PISA primer for fellow mortals. The international Organization for Economic Cooperation and Development, or OECD, gives the Program for International Student Assessment test – PISA – every three years to a sample of 15-year-olds around the world. The test of reading, science and math skills focuses more deeply on one of those three areas each time; the 2015 focus was science. More on that in a minute.

But first: math. (We'll save reading for another day, since it's strong here.) Yes, Massachusetts did better on average than many countries, and better than the U.S. as a whole. But math scores here haven’t improved in years, despite a lot of attempts to move the needle – and many of the nations with higher (and improving) scores don’t have nearly the resources that Massachusetts does.

“You guys ought to be the top of the top in the world,” says Marc Tucker, president of the National Center on Education and the Economy. He points out that the Massachusetts GDP per capita is more than $60,000 a year, and that the state has the nation’s highest proportion of adults with college degrees. And yet:

“You’re competing with four Chinese provinces,” Tucker says, along with other underdeveloped regions. “By any measure, poor countries – and very poorly educated parents compared to Massachusetts.”

And parental education – especially maternal education – is one of the few universally agreed-upon keys to raising educated children.

“What we’re looking at,” Tucker says, “is a situation in which a poor country is actually managing to do better than Massachusetts in math, and about the same in science.”

It’s true. Those four Chinese provinces, like Massachusetts, chose to be ranked as if they were countries – and did really well. So did Vietnam.

Now, it’s true that a lot of Vietnamese 15-year-olds weren’t tested, because they aren’t even in school; and it’s true that China doesn’t do much (anything, really) in the way of special education; and it’s true that there are many, many cultural differences, both in school and outside it, between Hopkinton, say, and Hanoi. Even so, you have to wonder what these countries may have figured out that we haven’t.

What Might Be Different Here

And that’s where it gets really complicated. But it’s also where PISA may be more useful than other tests. Rather than measuring students’ acquisition of specific curriculum knowledge, PISA is designed to test how well they can apply whatever they’ve learned to real-world questions and problems. That, at least in theory, reduces cross-cultural static when looking at the results.

Even more useful is the way OECD and other researchers use the data to develop theories about what helps students learn. Tucker has done a lot of this; the NCEE lays out some of its findings in the brief, and very readable, “9 Building Blocks for a World-Class Education System.”

The good news? “What we decided years ago is that Massachusetts is doing more than any other state of what is in those strategies,” Tucker says. Not so good, he argues: the selection and preparation of math teachers.

“I’d be willing to bet everything I own,” he says, that the pool of potential math teachers here “is nowhere near as strong as in China,” in part because China – like many other countries, but unlike the U.S. – requires aspiring math teachers to have deep subject knowledge and to take multiple math courses in college.

That’s true, says another specialist – but it’s true in elementary school, more than high school. And this is a test of 15-year-olds, don’t forget.

Grade-school math teachers are indeed “stereotypically not especially strong, even phobic” about math, says Jon Star, a Harvard education professor who researches mathematics instruction in middle and high school. “It’s getting better,” but the biggest challenge at lower levels, he says, “is that teachers don’t have deep enough math knowledge to teach the way we want to teach.”

At the same time, though, he says those teachers “typically are decent in terms of pedagogy,” or how they teach: using a lot of discussion instead of lectures, having students work in groups, doing projects – all the “student-centered learning” that researchers now say works best.

For high school teachers, though, “I would argue it’s the exact opposite,” Star says. “They, by and large, have decent content knowledge; they know a lot more math than the average elementary teacher. But it still is largely the case that these teachers are in front of the classroom, teaching students who are sitting in rows.”

And most reform efforts in the United States, even if they’re called K-12 reforms, “have really been K-6 or K-8 reforms,” Star says. “The assumption has been that if we solve the problem of what good teaching looks like in K-8, it will percolate up and solve high school. I don’t think that’s true.”

And Then There's Content ...

Ready to go a little further into the weeds? One issue is a long-running debate over balancing two key areas of instruction in math. Loosely speaking, it’s a question of Big Ideas vs. Procedures.

For a while now, Star notes, teachers have been told to focus more on the ideas than the procedures – the concepts, not the methods. That can work in the early grades, he says, but once you get to algebra, it’s a problem.

“There is a need to teach these very complex and abstract procedures,” he says, “and you’ve lost the ability to talk about what it even means to teach those procedures well” – things like showing students multiple strategies, asking them to come up with multiple ways to solve a problem themselves, engaging them in conversations about which methods are better and why, and which might be better in a different situation.

But high school teachers “have been told, ‘Don’t focus too much on procedures,’ and on the other hand are opening up textbooks, and they’re full of procedures,” Star says. “So they’re essentially rejecting advice they’ve been given, because it makes no sense to them.”

What to do? Star thinks it’s helpful to “nudge” high school math teachers toward new, more student-centered teaching methods, rather than expecting them to make huge changes overnight. And, yes, to learn from how other countries teach math.

“Chinese teachers tend to ask really good questions,” he says. “They’re also encouraging students to solve problems in multiple ways.”

He quickly acknowledges that there are cultural differences from one country to the next; China, for example, is “very exam-driven, high pressure.” But he thinks it’s always worth asking: “What can we learn from them?”

Math Isn't Just About Math

And learning more about what works in math could help in science, too. Again, Massachusetts did better than the U.S. as a whole, and there are other encouraging notes here, too: There’s no gender difference in the average science scores in Massachusetts, the percentage of students who failed to reach basic proficiency is smaller here than the world average, and (unlike elsewhere) the many students here with immigrant roots perform as well as their peers once socioeconomic status is factored out.

But the scores aren’t at the top, and they’re not really rising. Relative weakness in math could be a factor – you can’t do a lot of science without being capable in math. If Massachusetts students want to pursue careers in science, as many say they do, both math and science matter.

Commissioner Chester knows the stakes are high in an increasingly global economy.

“We’re not going to put the genie back in the bottle,” even at a time of “anti-globalization backlash,” Chester says. “It’s a very interdependent world that we live in.”