Tuesday, July 19, 2011

Atlanta -- Diagnosing the Nation's Testing Virus

Was it their fault?

Who is to blame when a whole rash of teachers and principals doctor student test results to make their scores look good to the district and the public?

The scandal in Atlanta schools has rocked the headlines with juicy details about teachers changing student test scores, administrators pressuring teachers to break the rules, and students with great scores one year followed by terrible ones the next.

Many teachers are probably thinking, “We told you so.”

While I would include myself in that bunch, I think we need to take this moment to analyze what this scandal says about our nation’s education system, on a deeper level.

“Teachers complained to investigators that some students arrived at middle school reading at a first-grade level. But, they said, principals insisted those students had to pass their standardized tests.”

Of all I’ve read about this scandal, this quote shines the brightest light on the problem.

Who To Blame?
First, though, we need a scapegoat.

The teachers? Well, when the person who evaluates you and decides to renew your contract tells you that meeting unrealistic and unreasonable test-score standards is the highest goal, and that we will break rules to do it, you have a difficult decision to make: Violate your conscience and go along with it, or fight it.

Fighting it could mean ignore it and hope your scores turn out well anyway, or it could mean standing alone as a hero, declaring that doing what’s right is the highest good. While that makes a good movie, most of us don’t have that kind of courage (maybe that’s part of the problem....?)

Ignoring it isn’t a bad solution though, right? If you’re a good teacher, you shouldn’t have to cheat. Maybe some people think that. But read the quote again. No matter how good a teacher you are, what are you going to do with middle-school students who read like first-graders? As I’ve said in other posts, expectations that are right for some students are way too hard for others.

If students come in reading like first-graders, the goal is to get them to read like second or third-graders. This is one reason state tests are meaningless gauges of teacher effectiveness and whether or not a student has learned. Not every student is at the same place.

So yes, while the teachers share some blame, they clearly were not the cause. So, who is?

The principals? At first, if we acknowledge teachers were pressured, it looks like we should blame the principals who pressured them.

Principals may run schools, however, but they don’t run them with autonomy. The principals were very likely being pressured by district administrators and the now retired superintendent. Get your scores up. You better not be a failing school, or we’ll fire you. I don’t know how they applied this pressure, but it was clearly being applied for it to be happening in so many schools.

In our district, this is called “performance management,” and every school gets evaluated based on a list of about twenty criteria. Some of these criteria are beyond absurd. Like attendance. Yes, in Seattle, our schools are evaluated based on student attendance. Because, as everyone knows, when students are absent, it’s the school’s fault.

So yes, while the principals share some blame, they clearly were not the cause. So, who is?

District admin? Well, using tactics such as these to raise test scores is contemptible, and these people must have made the choice at some point that ethics, integrity, respect, and honor meant less than scoring high and looking good to the nation. This is an example of horrible leadership, no doubt.

But, why did these leaders decide to do this? What precipitated their actions? What motivated them to compromise everything they should believe in? I can’t answer that here, obviously, but I can go one more level up.

So yes, while district admin shares a large part of the blame, they also were not the root cause.

So who is?

The Real Culprit
This scandal is a symptom of a virus that has infected our entire nation. It has seeped into communities, corporate board rooms, churches, both major political parties, and all the other “stakeholders” in education. Including parents and families.

Yes, these are all the ones to blame. Everyone who has bought into the flawed and destructive dogmatism that testing data is the authority on both student achievement and teacher effectiveness.

As I explored in the “Lunacy of High Expectations” series, we as a culture have this fantasy that students are things we can teach and make smarter, and that if everything goes right, they will put down all the right answers and prove themselves competent. We have ignored the fact that they are also human beings.

Developing a human being takes more effort than pushing buttons and filling in ovals.

We have elevated test scores to be the preeminent authority on whether our children have learned.

Perhaps you’re wondering, why is this a problem? If a student knows it, shouldn’t they be able to demonstrate that? Isn’t this what a test is for?

Yes, this is the purpose of assessments. But let me ask you a question:

Do you remember how to solve multi-variable equations? Do you remember who the generals were at Gettysburg? (Do you even know what Gettysburg is....)? Do you still know how to find the volume of a gas at a certain temperature? Or how to find the missing angle in a triangle?

My point is, if you don’t remember things you learned in high school, did you still learn them? Are you “educated”?

What does it mean to “learn” something? I have a class discussion about this in the first week with my chemistry students. One thing that often comes up is this question: If you have memorized something, have you learned it?

What do you think?

If I memorize all the elements on the periodic table, or every general’s name in the Civil War, have I learned?

While you think about that, consider that when a test is written, the question of how you demonstrate understanding is a different question than how you might demonstrate learning. I may indeed be able to rattle off all the generals, but do I understand what they were fighting about? Do I know the causes or outcomes of the war? Further, can I articulate that understanding in writing, in an oral presentation, or visually? And how much of this must I do in order to demonstrate understanding?

How much is sufficient to be considered “meeting standard”? Even more, who gets to decide, and why them?

Do you see the complexity? These simple examples are the same questions anyone who writes any test has to wrestle with. What do I want my students to know, and how will I know if they know it?

Standardized Infestation
Yet, when we read about national and state test scores, we never hear any of this.

What’s on the test? What kinds of questions get asked? Are they testing process skills or content knowledge? Are they testing basic knowledge or deeper understanding? Further, which of these should they be testing? Which deserves more emphasis?

And this is where we run into big problems.

I, as an extremist moderate, argue for the complete abolition of all state and national testing. I think it’s an absolute complete waste of time, money, effort, and manpower, and the more we emphasize its exaggerated importance, the more scandals such as Atlanta’s will come about. Further, I think it has little benefits for education as a whole and distracts us from our true mission.

But, in recognition that my desires are unlikely, let’s consider alternatives. The biggest problem: If we’re going to have testing, how do you test understanding on such a large scale? When so many teachers go at their subjects in so many different ways?

Example: What are we Testing?
Let’s take Writing as an example.

Suppose one teacher uses a curriculum she learned in a professional development seminar put on by the district. It has a prescribed set of lessons, uses certain books and short stories as tools, and concludes with the students writing a quality five-page essay analyzing the author’s intent in a certain piece of literature (poetry, short story, novel excerpt).

Suppose another teacher picks ten current events, and each month the students have to write an argument about those events. Each month the students get feedback on their essays, and each month their writing improves. By the end, the teacher has them writing quality five-page essays arguing for a particular opinion.

So, one set of students has learned how to analyze someone’s writing and draw out themes, arguments, conclusions, and other meaning. The other set of students has learned how to make a cogent argument on a particular opinion regarding current or historical events.

Now, suppose these students all take a state writing test, and the question is this: “Read the following speech from Hamlet, and analyze and explain three literary devices used by Shakespeare.”

Which set of students will do better?

Obviously, this is a hypothetical. But...oops. We didn’t emphasize that stuff. The teacher they’ll have next year is really good at teaching literary devices, and they’ll learn it all from her.

Meanwhile, don’t these two teachers look bad now. And yet, are their students learning? Are they building skills and understanding useful for their futures? Yep. Just not the ones the test writers decided were important this year. Last year, the question was about making an argument, and the second teacher’s students did great. What a coincidence....

What Is Education?
Yet, this over-simplified example isn’t that far off from one of the many complications exploited by the testing virus. Not all schools teach the same stuff the same years. Not all schools are even structured the same way. Some have block scheduling. Some have four classes, some have six, and some have eight. Some have trimesters. Some have “Language Arts” all four years of high school, while others offer specific course titles like American Literature, Creative Writing, and AP English.

Is any way better than another? Well, we can argue about that later. But the point is, not all students are being taught the exact same stuff in the exact same way at the exact same time.

AND THAT’S OKAY!

I say that loudly because some of the uniformity worshipers out there actually want all schools to do all the same stuff at the same time. It never works, and it’s unnecessary.

See, education is more than filling out tests. Education is about your ability to get things done. To adapt to new situations. To think and analyze ideas and information. To reason. To meet deadlines. To grow in understanding about new ideas. To develop skills.

The truth is, there is no single test for all this. It doesn’t exist, and it never will.

Testing is incapable of determining whether a student should be able to graduate. Testing alone cannot determine this.

Why Testing Won’t Work
The truth is, people place far too much hope in education to solve all the world’s ills. I mean, how does a child show up in middle school reading at a first grade level? First grade! Think about that. What combination of factors has produced this child, and what possible hope do they have already being this far behind? And how many other students are like this? And how is testing going to make any difference for them?

At my high school, our freshmen show up with an average of a sixth grade reading level. So we have a year and a half to raise them four grade levels so they can pass the state test. You better hold us accountable!

We are told poor education is the reason some communities continue to struggle with poverty.

No, poverty is the reason some communities continue to struggle with education.

Broken families are the reason some communities continue to struggle with education.

Misplaced priorities and materialistic and chemical addictions are the reasons some communities continue to struggle with education.

Think about that one. People get into massive credit card debt, for example. Money gets tight. Debt collectors start calling. Stress increases. More arguments among parents. Less stability for the kids. The future begins to feel uncertain. School becomes harder. Maybe it leads to divorce, or maybe a lower standard of living. Maybe they have to move. Again.

Instability and uncertainty are poor friends with a good education.

The chaos I’ve witnessed on reality TV shows about parenting is enough to remind me why education doesn’t improve. Can you imagine having some of these people as parents?

But the testing fanatics think that as long as scores go up, learning has happened.

As Atlanta teaches us, if scores go up, that just means scores have gone up.

Even in districts that don’t actually cheat, every district and every teacher in some way feels pressured to teach to the test.

We spent two weeks teaching after-school test prep sessions for the science state test this year. We used previous state test items as examples, and prepared the students for the goofy ways they phrase and structure questions, helping them to see how to tie in to the knowledge they already have so they can successfully jump through the hoop (because that’s what it is, and everyone–including the students–already knows it).

And when the actual tests came out, guess what happened? They changed all the test question formats. According to students I spoke to, most of what we covered in the test-prep sessions wasn’t even on there. Two weeks of work. Over 30 combined man-hours outside of school.

It was a total, complete, absolute waste of time.

And I will never do test-prep again.

Even if you paid me for it, you merit-pay proponents. Even if you paid me.

Because I know if my students have learned, and I know how to write my own tests. And if you don’t trust teachers to be able to assess their own students, then you should start a campaign to eliminate grading, because this is what grades are for.

(But then, if you don’t trust teachers to assess their own students, how in the world can you possibly trust test-writers and bureaucrats to do it–people who aren’t even in the classroom?)

Who to Blame?
So who’s to blame for what happened in Atlanta?

First, yes, the people who actually did what they did. But that’s easy.

The real culprit, the initiator of the infection that has now shown its symptoms in Atlanta, is anyone who has pushed testing as the means to evaluate whole systems of students and teachers. The real culprit, then, is you, if this is something you’ve championed.

This has been years in the making. I remember in the 90's when standards were just becoming fashionable, when I was just getting into teaching. Even then, the idea of a state test that everyone had to pass to graduate struck me as odd, and I hadn’t even taught yet.

Now I’m sure of it. This is a bad idea. It’s a waste of money. And it’s helping no one (except the testing industry).

My new idea? Let’s make the testing craze benefit more people. Can we convince Las Vegas to give us odds on which city will have the next testing scandal? My money would be on any district that evaluates teachers and schools based on test scores.