John Jensen: How Testing Can Ruin the Matrix

National testing of educational outcomes goes back a long way. 1964 marked the beginnings of the National Assessment of Educational Progress, whose application has expanded ever since. The idea was not to force people to comply with anything, but rather to provide a comparative glance at progress in key areas. The plan was respectful in assuming that people of good will would use valid information appropriately.

John Jensen, Ph.D.

So what changed in 2001 with the passage of the No Child Left Behind Act?

Its admirable intent was to increase students’ performance, but with a new assumption–that information developed state by state could be leveraged, that with it you could make schools and teachers do what they apparently would not do on their own. NAEP information was regarded as insufficient to effect change.

The Feds tried a strategy to sidestep the protest likely to occur if they attempted control from the Federal level. They shifted the hands-on stuff to the states: Find out how your schools are doing, but Federal funds will be the consequence. They presumed that states could test their own learning validly enough to certify “adequate” progress, but set this to drive a vast consequence system.

One spots possible weaknesses in this reasoning. Are districtsreally unwilling to take up effective means of education without being forced? Is the educational system really motivated by financial pressure from Washington better than by any other means? Is testable learning assessed under such conditions really valid? Ten years of results from the NCLB reveal skepticism, showing up in resistance, avoidance, bending, cheating, and argument. We have not heard general acclaim that “This is the solution we’ve been waiting for.” Pushback instead might lead a vendor to think, “The customer does not want to buy what we are selling.”

We could imagine a teacher and an education planner arguing about the role of summative testing in the prescribed plan:

Teacher: “Testing as you organize it demands that we teach to the test in a way that overall learning suffers.”

Planners: “But testing is the only way we can know what students are learning.”

Teachers: “We already know what students are learning, but for you to find out the way you do means that students learn less. Is that a tradeoff you really want?”

Planner: “When we started, national learning was so impaired that we had to test in order to know what to do so students would learn more.”

Teacher: “You’ve been getting voluminous information for years now. Since education still struggles along, your premise must be flawed. You misjudged what test information would enable you to influence.”

Part of the resistance may arise from an unvoiced message implied by the plan, a judgment of teachers. It presumes something about them, telling them how Federal planners regard them: You are unwilling to do the right thing on your own. We think you respond to the threat of sheer financial loss, so we’re going to try that. You’ll improve or suffer consequences. If this is what those with political power think of teachers and if they’re right,should we be trusting teachers to form our children’s hearts and minds? Your own children–would you want them taught by people who need financial pressure in order to change? The implied message is an insult, and the overt message is threat of financial harm. With these two clubs hanging over them, we then send teachers into the classroom to focus on learning.

To unwind what happens next, we’re limited by a facet of human consciousness. A crucial quality of it is how our attention narrows around the task at hand. When a person does something, we can assume that 90% of their attention goes to the thing they’re doing. Though we may initiate a routine task and run it with 10% of our mind, we reserve 90% for one containing variables. The umbrella concept we want 90% of a teacher’s mind absorbed in might be expressed as Children advancing. We want them focusing on where children’s knowledge and attitudes are, and taking them the next step of development.

Assuming that’s what we want, how do we modify that message when we add, “…and by the way if you don’t do well enough, you’re going to suffer financially”?What changes in the teacher’s mind? A research study about this might surface objective evidence about unintended consequences of the current plan. Thought processes matter immensely, and major shifts in them alter outcomes. While awaiting more evidence-based insight, a couple of hunches might get us started:

The predictable influence on their thinking is that the structure of the plan mandates them to pay attention to the financial consequences of their actions. Follow me here. The Feds intend financial consequences to be the motivating factor! They have said, “More altruistic or caring values don’t move you, so we want you to think about your own potential loss.” The plan declares, “We are certain that if you keep thinking about the consequences we direct at you, you’ll teach better!” Implementing this direction then guides how a school frames its purpose: We want to produce student answers that score well on the tests that affect our well-being. Adopting this limitation is not a teacher perversely rebelling against a reasonable direction, but rather doing what the plan asks him to do.

Much channeling of effort, of course, is done by the school’s curriculum that directs activity toward what will be tested. The teacher then must get students “from point A to point B.” The first is what students don’t know now, and the second is what will be tested. Teachers are asked to help students give the dollar-driven answers on the test. Teachers who consider themselves professionals insist on thinking differently, refer to their own values, and teach more broadly, but even they are constrained by the selection and pace of the curriculum.

Summed up, the shift is in systemic emphasis—from learning advancing to learning accounted for. An emphasis asks people to keep it in mind. It becomes a focal point, displacing everything else because the mind is not adept at sustaining two important things simultaneously. It’s as though in designing an automobile, we fill the entire dashboard with a blinking sign, “Policeman ahead!” Yet knowing our car’s speed is incidental to the purpose of the car. We don’t want to be so riveted on the blinking sign that we cease looking where we’re going, but that is often what happens.

Thinking reactively drives out thinking intentionally. Results that hit your pocketbook, even indirectly through your district, are guaranteed to seize your attention. “High-stakes” outcomes drive you to accommodate to them, and facing a pending assessment, you do the kind of thinking enabling you to respond to it,while deferring or minimizing thinking you might otherwise prefer. What goes missing is a kind of thought structured differently than usual testing elicits.

Constructing knowledge. A way to regard learning is as a constructive process. Our senses pick up incoming stimuli, and then a mysterious internal capability somehow converts physical impulses to conscious information.The mind saves up the information by two basic activities. It links and repeats. Taking up a newly formed chunk of knowledge, it ties it to something else, goes over it again, and continues these two activities—linking and repeating.

Linking the information turns individual chunks into a coherent network.A thread of meaning common to the network holds the pieces together long enough for the mind to move smoothly from one to another along their logical linkage—generating understanding. Since such traces upon the mind tend to evaporate unless repeated, we also repeat them—explaining and expressing them again and again. This combination of linking with repeating fixes the pieces in an ever-broadening field,an array of knowledge that possesses an inherent unity, and at one level can be grasped in a single glance. Students can respond to an invitation like “Tell me all you know about math,” or “Tell me all you know about people around the world,” or “Tell me all you know about science.”

Linkage occurs at several levels. Imagine biology students spending their first monthlearning the table ofcontents of a 600 page text. They memorize the actual table itself (deepening their grasp of it by repeating it), but also talk through the organizing concept of the table, how the major parts relate, how the sections belong within the parts, and the kinds of details students will encounter later.An initial template waits to be filled in and presents hooks of meaning where details naturally alight.

A parallel is a landscaper hired for a new subdivision on a hillside. Consulting the plans, he sends out bulldozers to cut and fill what later will be home sites, lawns, and streets. Similarly in education, by first grasping the fieldin which later learning is placed, with every new detail added you reinforce and enhance the field-knowledge,and more accurately attend to the detail. Linking pieces into a network means having conscious possession of how they inherently go together, a comprehension obtainable only by progressively explaining the field.

Linking takes a step beyond collecting the pieces. One can study chapters in the biology book as though they were different subjects—about Russia, sea urchins, or astronomy. Taught in this manner, one cannot lean on understanding to grasp the subject but must rely completely on memory, lacking a sense of the internal relationships skipped over.

For the contractor, linking in a cursory, superficial way is like having separate pictures of a swimming pool, a lawn, street signs, a fence, and underground pipes—while knowing nothing about the subdivision. This picture is a lawn, and that one underground pipes, but without their common field, they do us little good. We value the individual pictures because they go together into a coherent image of a completed subdivision.

Facts about biology have value because they enable us to make sense of a field within which they contribute meaning. Their stand-alone data help little, are only a means of filling out a larger picture. If our intent is to comprehend biology, we take the trouble to place the pieces in a field of understanding so that a student who passes our course can think biologically. You want him able to answer a field question, like “Now apply what you know to farming,” or “Explain the oceans as a biological system,” or “Relate biology to human diseases.” No ready facts answer those questions. The mind must sort and sift within the field for associations and links to develop a response.

You can tell whether ideas are linked or just lumped by how you think about them. With pieces linked intrinsically, you logically or associativelyproceed from one to the other. One somehow suggests or inheres in the other, or an elaboration of the prior leads to the subsequent. When pieces are organized instead just by “being math” they can stand as much apart in the mind as the objects you find in a random kitchen cabinet; maybe pet food, paper towels, and skewers—all kitchen-related– but knowing about the pet food doesn’t lead you to the skewers. You have to rely on memory.

We return to the linking function of the mind to createfield-knowledge. The components accumulating may be visual, structural, sequential, sensory, auditory, kinesthetic, and social. Rich knowledge is an internal field with multiple strands extending from it. Pick any strand, follow it, and you have a smorgasbord of associations—which is how we prefer to hold onto knowledge.

And why do we form such a field? Apart from utilitarian reasons, we do so because it interests us. We learn things that we don’t use to make a living because it interests us. And even something we learn that initially didn’t interest us may become so just because we draw it from someone else, just because something comes together for us, something jells. We get a flash, an insight; notice a trail to follow—and the pursuit of it affords pleasure.

This multi-faceted representation of knowledge shows up in how people think about their occupation or any personal predilection. Pick a person, find out what they do, let them know you are fascinated by it, lean back, and watch what happens. Their mind will effortlessly proceed from one idea to another, following chains of association. They will spontaneously proceed along rankings of importance among details, narrate how their physical activity ties to their purpose, make personalized references, and incorporate social meaning. They will explain the challenges in their activity, and how they meet them.They will let you know what it is about their work that, when the day is done, they look back upon with satisfaction.

All of this rich, related data is afloat in an intensely-linked internal field sustained by how they explain it to themselves. As their field of attention expands to incorporate more actionable elements, their competence expands. Like someone taking a test in school, they can accurately explain in detail what they are doing, and their actions are effective.

The teacher example. The power of such field-related engagement with the details of one’s work shows up in the teacher axiom, “You learn a subject by teaching it.” Why should that be true? What is the difference between what you thought you knew when you graduated from college, and what you know you know after teaching it a few years?

A way to describe the difference is the integration of a field. Where effort at first went to remembering details, repeated treatment of them left you so confident of them that you could turn your attention to aspects of greater interest, pursuing nuances of meaning emerging from the larger field. When you explain something repeatedly, your mind steadily apprehends pieces in an altered shape, with different connections, in more effective words, with novel examples. At the same time, the more basic structure of the field deepens, so that by returning to the same points you fix them ever more securely in the most grounded part of the your thinking.

Your own growth while doing this offers a clue about testing. Could we agree about a motorist, first, that once he’s within the speed limit, the dashboard blinking sign offers him no help at all? And that the financial pressure bore no relation to your own development? You expanded your field probably by just three efforts: a cursory review of old material to affirm what you already knew, noting what you wished to add, and then expressing it again and again. All your learning came through that pipeline: get it in your head and express it again. Only after you had done so far more extensively than anything expected of you when you first learned itdid you even think of yourself as having mastered it. Mastery came from repeatedly and comprehensively explaining it.

A field using class time. Hold onto that picture as we examine typical class time for the two processes short-changed. The first is that the field is never expressed.In their entire K-12 schooling, students never hear the invitation, “I insist that you learn this entire subject and be able to explain it beginning to end.” Assignments and teacher input alone do not form a field.It develops instead by our selection of our own words to link together the pieces we have independently acquired. Creating the field piggybacks on selecting the words that describe it.

Teachers typically approach this field knowledge by discussion, stories, projects, student presentations and reports, and hands-on experiments—activities seldom reducible to bubble answers. But with the high-stakes mandate gobbling up attention, these activities are the ones short-changed so that the field is not constructed. The matrix is missing. Ideas are not laced together because they don’t help student answers on the test that will affect district finances. Pressure to produce testable answers displaces the time and focus needed to create the kind of learning we value most–comprehensive, mastered field knowledge, a matrix comprised of the knowledge itself.

Is the tradeoff worth it?

John Jensen is a licensed clinical psychologist and education consultant. His three volume Practice Makes Perfect Series is in publication with Rowman and Littlefield, education publishers. The first of the series due in January is Teaching So Students Work Harder and Enjoy It: Practice Makes Perfect. He welcomes comments sent to him directly at jjensen@gci.net.

John Jensen is a licensed clinical psychologist and education consultant. His three volume Practice Makes Perfect Series is in publication with Rowman and Littlefield, education publishers. The first of the series due in January is Teaching So Students Work Harder and Enjoy It: Practice Makes Perfect. He welcomes comments sent to him directly at jjensen@gci.net.