#66: What does standardized testing measure anyway?

by mches

If you will forgive the drought between posts, I did actually begin this one last month and it’s been sitting in the drafts folder ever since. But I think today is as good a time as any to revive it due to the latest legislative news. Among other things, House Bill 5 eliminates the requirement to pass many of the state’s end-of-course exams (STAAR) for high schoolers. Previously students were required to pass three years of testing in reading and writing (English 1, 2, and 3), math (Algebra 1 and 2, Geometry), social studies (US History, World History, World Geography), and science (Biology, Chemistry, and Physics). The requirements have been reduced to combined writing and reading for English1 and 2, Algebra 1, Biology, and US History.

As a practical matter, this frontloads most of the required tests in students’ freshman years, giving them more opportunities to pass it should they not do so the first time around. It is, however, still high-stakes (pass these or you do not graduate). To my knowledge, this bill does not change how campuses are graded which has typically been how a high school campus’ 10th graders fare.

—–

If you have not done so already, please read this piece in Texas Monthly by Nate Blakeslee. This goes double for teachers in Texas. It does an excellent job charting the rise and fall of standardized assessments in Texas and how our state has influenced national policy. Texas is the birthplace of test-based accountability. It worked so well for then-Governor George W. Bush that he imported the approach to Washington as president with the No Child Left Behind Act.

Since then, educators have been engaged in a quixotic battle against reformers and test publishers. One of the biggest bones of contention is the culture that high-stakes test-based accountability encourages. Academically, this means a narrowing of the curriculum by dint of the core content area getting stripped down to what will be on the test(1) and/or emphasizing only what subjects are tested(2). Defenders of test-based accountability argue that the practice of teaching to the test is a bad practice anyway, that if schools wanted to improve test performance, they would simply teach the full scope and rigor of the curriculum. You see this in the words of Bush adviser turned Pearson lobbyist Sandy Kress:

Later, on an education blog, Kress elaborated on his response, rejecting the notion that high-stakes testing was turning Stanford’s third grader into a professional test taker. “I can hear the cry already: ‘they’ set such high stakes for the test, ‘they’ have ‘made us’ do these stupid things,” he wrote. “No! Nonsense! The tests should have consequences. But drill and kill is no solution. Good teaching to the standards is the only solution that works, and I refuse to be identified with the crap that is done instead.”

Or, more close to home, Gary Rubinstein sparring partner Matt Barnum had this to say in the comments of his back-and-forth:

Though the truth is, I am not too concerned about ‘teaching to the test’ – I think to a large extent teaching to the test is a good thing, if the test is adequately designed.

Or, take scholar Eric Hanushek. In his correspondence with Deborah Meier, he similarly sees no issue with teaching to the test, as long as the test covers everything:

I think nonetheless we agree that these test scores are narrow and rudimentary, generally failing to measure the higher-order skills that are increasingly seen as important. Moreover, considerable attention has been given to the problems of teaching to the test, to the occurrence of cheating episodes in different cities, to possible distortions in teaching that come from the testing regimes, and to the application mainly at the bottom end… And now the point—I don’t think that we have to be stuck with the current problems with our testing…It starts with developing a large item bank of test questions of varying difficulty. Imagine 2,000 questions for 4th grade math that cover the entire scope of appropriate material from basic to advanced topics. Next, make all of the test items—not just sample items—publicly available and encourage teachers to teach to the test, because the items cover the full range of the desired curriculum.

Imagine, 150 4th graders in a computer lab(3) being drilled on those 2,000 questions all year. It’s an edutechnocrat’s dream. What I take away from this is that test-defenders can never be convinced of the problems inherent in high-stakes testing. Testing can never fail, it can only be failed. If testing is bad, either the test is too easy or it is implemented in the wrong way or designed poorly but somewhere out there is a test that does exactly what it says it does and has no confounding factors whatsoever. They believe that schools that practice “teaching to the test” do so at their peril and not to their advantage. After all, schools that do not teach to the test tend to perform better on tests anyway.

Let’s unpack all of this. Why do schools that don’t drill-and-kill perform better on tests? I’m going to take a stab in the dark and say it’s because schools with low-poverty demographics have historically not had to worry about high failure rates to begin with. The way Texas’ accountability system is set up is to compare a few disaggregated demographic groups on select state tests. High-income schools have higher-performing students overall due to lifelong socioeconomic advantages, so all their subpopulations do well as the campus does. The level of rigor of tests like the TAKS(4) were such that high-income schools had relatively easy times getting nearly all their kids to pass their exams.

Low-income schools, though, feel the perennial threat of being labelled “academically unacceptable”. Too many of our students are at risk of failing these exams and thus bringing adverse consequences to the school. Because of how important the results of these exams become and because of how far behind their wealthier peers our students often are, schools like ours often resort to “drill-and-kill” techniques designed to get kids ready for the tests.

Furthermore, chasing this idea that a single test can neatly summarize the year-long learning of every child is a fool’s errand. Hanushek’s lament is the most revealing(5).

While I am grateful that my students now have a less unreasonable path to graduation, I hope parents and educators and students do not consider House Bill 5 the end of our attempts to have richer educational experiences for our students.

NOTES

(1) i.e. Why would you teach your kids constructions in Geometry when you know that they will not be given a compass and be graded on constructing a segment bisector?

(2) i.e. Why would you offer a panoply of elective coursework when so many kids failed the Algebra I and Writing exams? They very obviously need more remediation and won’t have time for art.

(3) Please read this, it is incredible.

(4) Our old state assessment. I was in high school when NCLB was first implemented, and my graduating class was the last one to take the TAAS test before they transitioned to the TAKS. None of my peers gave a rip about these tests and none of our teachers talked about them at all. We had our testing day, we took the tests, and we moved on. TAKS was a big deal for a good decade until someone got a fool notion that the tests were too easy and they dreamed up the STAAR exams.

(5) Not at all surprised that Hanushek is an utter educational charlatan. I’m a little surprised he would make it so obvious by proposing year-end tests be publicly available and believing this would reduce drill-and-kill.