Testing the limits of testing

Testing the limits of testing

High-stakes testing in K12 schools has had a chilling effect on how students are taught and what they learn.

The number of school closings due to poor test results has nearly doubled annually, from about 1,100 to 1,900, notes journalist Anya Kamenetz. Likewise, the number of school reorganizations has spiked from about 1,000 a year in the mid-2000s to about 6,000 a year in 2010-2011.

A contributing writer for Fast Company and the author of the acclaimed DIY U: Edupunks, Edupreneurs, and the Coming Transformation of Higher Education, Kamenetz says the fallout from test scores goes deeper than most realize. “It really only takes one school to be threatened with closing to destabilize an entire urban district,” she says. Her upcoming book, The Test (to be published in 2015), is an in-depth look at the history, impact and future of educational assessment.

You wrote an article this year about a school whose students demonstrate connected learning in group projects, but they’ve had pushback that students are not doing enough book learning.

I think we’re living in two worlds, if you will, in the sense that there’s a lot of budding interest in creativity, the importance of higher-order thinking skills, critical thinking, communication, and collaboration associated with success, even more broadly than just learning. At the same time, we have this very entrenched apparatus of accountability in assessment that is based on a pretty narrow conception of academics and tested skills.

There’s so much accountability and rewards and punishments attached to the existing tests, that it makes everyone nervous that if we go too far into a creative concept of learning, teaching and testing we’ll fall behind in the assessment areas. Quest to Learn, the New York school in that story, is an example of a school that tries to live in both worlds.

Within the framework of games, I think what’s interesting is that you learn to play a game by playing. Failure is very much part of the process. You die, you get another life, you start over. It’s the idea of incorporating failure as an interaction and as a reason to try again. That’s so powerful for kids, especially kids who have been subjected to this idea that if you go in on a test and you don’t make the grade, it will have negative consequences forever.

This sense of failure really permeates. The fear is magnified from the teacher and district, to the state and federal level, and down to the level of a third-grader.

How did we become such a test-obsessed society?

It’s such an interesting question, and I did have to delve pretty far into the history of intelligence testing to answer it. We really are interested in science. We’re interested in rational thinking. We like to have numbers and we like to have measurements.

Measurements are a great way of making decisions, but you have to be careful that you are actually measuring something that’s real. The problem with tests has always been that human intelligence is extremely elusive and any measurement of it is going to be just an approximation. When we ascribe precision to these measurements, they may be precise, but they don’t necessarily tell us anything important about the real world.

The sinister side of the obsession with standardized testing is very deep and old as well. The people who developed intelligence tests—almost to a man—were interested in superior and inferior mental ability, what it meant about eugenics, and the sterilization of the feeble-minded. To me, that stems from a very deep distrust of difference and of diversity.

I wasn’t aware of that history.

It’s really fascinating. Alfred Binet, who created the first practical intelligence test, is an exception, but many of the different figures who contributed to the founding of the SAT and who invented statistical techniques such as the correlation coefficient and factor analysis—these guys were eugenicists.

Francis Galton coined the term eugenics and he was one of the earliest psychometric superstars. This is well-documented history, but it’s not something we talk about a lot when we discuss No Child Left Behind.

Students today spend months learning the test. Teachers hate it, the schools hate it, but they are forced to do it. Did you hear that often?

Oh, sure. It’s very real. Students are taking up to 25 percent of class time preparing for tests. And there are all these extra benchmark tests that kids take—field tests and practice tests. I would estimate that almost three years out of school lifetime is spent on taking these tests.

Clearly it distorts the curriculum. We have evidence that it causes teachers to spend far more time on the tested subjects because they want the kids to do well. It’s a very difficult balancing act for teachers to try to promote deeper learning and creativity when students are mandated to perform well on these narrow measurements.

Elementary and high school students have just begun taking the Common Core tests in 36 states. Is it ready for primetime?

It’s disappointing because, however you feel about the standards, the tests just don’t measure up. Linda Darling-Hammond, who was part of the Gordon Commission—a respected independent commission that reviewed the PARCC and Smarter Balanced tests—told me these are basically not that different from your standard old multiple choice.

And you can tell that by the fact that they’re not that much more expensive to implement. If you are going to create a test that really measures deeper learning—or fewer, higher, deeper, as they say—you have to invest in it.

You have to create performance tasks that involve lots of long, free-response questions that are graded in lots of different ways. You’d want tests that reflect fluid intelligence and divergent thinking. But these things take time to grade and they are difficult to administer.

That’s not what we have. The Common Core tests are pretty much the same old thing, dressed up in new clothing.

I’ve heard educators say the new test will result initially in lower scores, but don’t panic. Yet people are panicking.

Sure. We’ve been conditioned for so long to accept that these proficiency scores actually tell us something meaningful about how our kids are doing or how good a school is doing. If you are going to now come in and say the new scores don’t mean anything, then it pretty much undermines the whole process.

What’s your opinion of the PISA test?

The role of tests like PISA is really interesting. They are given to super-large numbers of kids around the world, basically for benchmarking purposes. They’re not high-stakes. I think you can have a lot more confidence that, at least on a comparative basis, they’re giving us some useful information.

Can the United States ever rise to the level of Finland or Korea on that test?

We’d have to really recalibrate our national priorities. There’s a growing consensus that the most important input toward student achievement is child poverty and all the factors that go along with it. Unless and until we recommit ourselves to ending child poverty and reducing inequality, not just in schools but in families and communities, I don’t see us reinventing ourselves.

We can’t put the cart before the horse. We’ve spent too long at this point saying, “If we improve schools we’ll erase poverty.” I think it’s really time to try the other way around.

Will your new book touch on the SAT?

Yes. The recent reinvention of the SAT, to me, was a competitive move in some ways. For the first time, SAT has lost ground to the ACT, which has successfully marketed itself to school districts in a lot of states.

But in the bigger picture, these tests and all the tricks and traps that go along with them, are not that important. They are not, independently, particularly very predictive of kids’ success in college, which in turn is not necessarily that predictive of their success in life.

The SAT was revised in 2005 with an essay section, and now that has become optional. It seems they don’t know what they’re looking for.

Yeah, it’s pretty pathetic. One of the things I talk about in the book is the importance of assessing expressive language. That’s a key skill for employers, for life, writing and speaking. Most standardized tests don’t do a good job of that. The reason for that is mass testing is mass graded, and the grading of essays is not very good quality.

There’s no easy fix. The type of testing that we’ve incorporated in schools has been marked by a mass production mentality. We test what we test because it’s easy and cheap to test it that way. Until we move toward a mentality that’s a little bit smarter about it, we’re not going to get anything valuable out of these tests.