By All Measures: The 'Most Promising Way' of Getting the Education We Want

MS. RESNICK: It's tempting to just take up Ted Sizer's questions one
by one, because they are so important and he poses them with so much
clarity. I can't do them all in the limited time I have, but I will
address some of them.

Is changing tests, changing assessment, really the best lever for
reform we can imagine? I want to suggest why it is, why a serious
effort to change the assessment system under which we now operate might
be our most promising way of getting to the kinds of education we all
want.

The main reason is that we already have, in effect, a national
testing system, based on the kinds of grade-level data that come out
once or twice a year in every school district around the country that
show whether the system as a whole, the individual school, and
sometimes the individual class in a school, are performing up to grade
level, above it, or below it. That constitutes our national testing
system. That's the heart of it. We cannot pretend that we're starting
with nothing, and now we're going to move to some new system with all
of its inherent risks.

There's something else much more interesting, much better, called
the National Assessment of Educational Progress, which is hardly in the
news, hardly in the play. It happens only once every several years,
and, at least up to now, nothing much depends on what the results are.
We use it to monitor how the system as a whole is going. Many people
don't notice those scores; many teachers have no idea what
NAEP is.

The testing system we have that really makes a difference are tests
such as the Iowa Test of Basic Skills and the California Achievement
Test, the Metropolitan Achievement Tests, and a couple more. These are
our national testing system. Most of our kids take those tests at least
once a year, sometimes twice. And if they're in any kind of special
program, several times, very probably. And, what's more, subtly or
overtly, teachers, principals, and school superintendents are held
accountable for how those test scores look.

What's wrong with the present testing system? Why is it worth doing
the New Standards Project and taking all the risks inherent in a new
national system of assessment? Why don't we just stick with what we've
got?

Well, what's wrong is that those tests provide an absolutely
terribly model of the kind of learning we want.

They are collections of items that you have to answer at the rate
of, at the slowest, one per minute, in order to do well on the test.
The passages that a student traditionally reads in this respect tend to
be 500 words; that's two typewritten pages. That's what we consider the
complexity of text that is the most complicated that our 8th to 11th
graders should be reading, as measured by our de facto national testing
system.

The mathematics items are even worse. And we measured writing, until
very recently, by one's ability to correct spelling, to check spelling
errors, and so on. And to do that at the rate of about two items per
minute. That's what those tests look like.

No. 1: We've got a terrible model of what knowledge is, and what we
care about, built into those tests: Collections of decontextualized and
decomposed bits of knowledge that do not add up to competent thinking,
to knowing a body of history or science or whatever we might care
about. We have the assembly-line version of knowledge: Break it into
little bits so any nincompoop can fill in the bubbles.

No. 2: We have tests that have built into them the theory that how
far you can go is all in the genes and maybe in the first three to five
years of family life. The tests spread kids out on a curve; the items,
in the end, are selected to enhance that spreading out. All of the
items on a math test do have something to do with arithmetic or math,
but the way the tests are built, fundamentally, is by setting up a pool
of a hundred, maybe a couple hundred, items, and trying them out on
kids. From that pool, items are selected that will spread the kids out
the most, because that's what will enable them to be assigned to
certain percentile ranks most reliably.

In effect, all the items that everybody can do, and all the items
that almost nobody can do, are thrown out. But those are the two that
are the most interesting in many ways. The worst thing you can imagine
is throwing out those parts of the test that show that you've
succeeded, and that show kids and teachers that they can succeed.

You also don't want to throw out the ones that are making so much
trouble that hardly anybody can do them, because, in a way, those are
setting the stars to reach for, the targets that we'd like to get to,
even if we can't do it yet. Or, those we'd like some people to get to,
even if not quite everybody.

This is all elegant technology, and it's done very honorably by the
testing companies.

But the message is profoundly debilitating. If you start out in the
70th or 80th percentile, either as an individual student or as a school
that's blessed with students who come in able to perform that way, you
sit back and say, "Well, I don't really have to work; there's nothing
much more for me to reach for, I'm already in the top 20 percent or 30
percent.'' And if you start out in the bottom, let's say the 30th
percentile or the 20th, the basic message, once you've come to
understand it, is that you can never get out. Because the only way you
can rise from the 30th percentile to the 70th, say, in real terms, is
for everybody else to wait around for you to catch up, and even let you
surpass them. Well, that isn't going to happen, nor would we want it
to.

It isn't too likely that a child or a group of children who start in
the bottom quarter are going to end up in the top quarter in a
comparison. But they sure are likely, if they work hard and have a good
curriculum and have teachers who are empowered and thoughtful and
working hard, to learn a lot. And that's what we want to be able to
show them. And that's what today's national testing system makes
impossible.

So, for those two reasons, this present national testing system has
to go.

If I thought I could get rid of it by waving a wand, I would be
working in ways other than on assessment as a lever for
change--investing heavily and directly in staff development,
curriculum, and the things surrounding both of those.

But because I think there is zero political chance that this
country, any time soon, will give up its current national testing
system without having a replacement for it, the only way to get going
on what we need to do is to attack directly what is one of the most
powerful dampers to the kind of change we need. Talk to teachers who
have caught on to the idea that the kind of teaching required in a
"thinking curriculum'' is possible, and then ask them, "What is the
biggest barrier to it?'' Their answer every time is, "Those
standardized tests are coming, and I'm afraid my kids won't pass
them.'' It takes courage of an extraordinary kind to say, "I'm just
going to take the risk and forget about the tests.'' The pressures to
drill to the test are overwhelming, and they are overwhelming mainly in
the schools that serve our poorest children.

So, I now reach the equity part of the argument.

In our more privileged schools, first of all, the kids come in doing
pretty well. They don't need to really worry too much about those test
scores. There's some worry, but it's not overpowering. And so there's a
certain liberation.

But in addition, in those communities there's an implicit knowledge
about what's really worth knowing.

In the inner cities, however, where the accountability burden is
enormous, and where there is not an implicit knowledge spread through
the community of what the alternatives are that really matter, the
drill to those bubble tests can be overwhelming and stifling. There's a
double standard built into our current national testing system that
holds poor kids, children of color, language-minority children, to a
standard of what it is that's even worth learning that is different
from the implicitly held standards of the more-favored schools.

All of this constitutes what I consider a totally unacceptable risk
to take: Namely, the risk of continuing our current system.

I am willing to take the risks, some of which Ted alluded to, of
trying to move toward another system, because I think staying with the
current one is more risky. We already know what's happening as a result
of it. So, it's worth taking those risks to forge something better.

As Marc said, this isn't about exams; it's about improving education
and learning for all kids. The way to make that happen is not by asking
how do we build a better test and then hoping people will somehow get
kids to do better on it. The way to make this happen is to ask: "How
can we embed an internalization of these new goals and criteria and
standards for learning throughout the education system?'' And that
means primarily among teachers. We think the only way to do that is by
having teachers--and maybe principals and maybe curriculum supervisors,
but largely teachers--be the primary agents of the whole process.

The New Standards Project is committed to the position that children
will not be taking assessments unless their teachers have participated
in the building and scoring of them. And so, in a process that has
begun in the past year and will proceed in a kind of pyramid or
training-of-trainers model, we are working with teachers in the 17
states and half a dozen school districts that are participating in the
project.

They develop the tasks in an interaction with people from the
national disciplinary organizations like the National Council of
Teachers of English, the National Council of Teachers of Mathematics,
and so on.

We are not test developers, and we are not ourselves standards
developers; we are doing this with the nation at large. The teachers
develop tasks, and they pass them back and forth between the
subject-matter and curriculum groups, and eventually come to the kind
of pilot that went on a few weeks ago with 12,000 children around the
country participating.

The teachers, in so doing, are engaged in reviewing and even
creating content standards. They do the scoring. These are matters for
human judgment; to score the tasks Marc talked about, you have to
understand what the criteria are. And our teachers participate in
developing those criteria.

The discussion surrounding that scoring is a massive piece of
professional development, and we intend to study that process. We know,
in fact, that virtually any kind of technical scoring can be made
adequately reliable. The question is: Which kinds of scoring activities
flow back into teaching with the most power? That's the question that
matters. We can get adequate test reliability using any one of the
technical methods that are around.

So, all of these things are part of professional development.
Teachers are doing just what Ted said; they're looking at student work.
There's a massive reliance on examples as the only serious way to set
standards and build an understanding of them. And they will participate
with other teachers.

All of this impacts on how one even wants to think about the cost
question. We have commissioned some work by economists to study actual
costs, and we'll have something by the end of the summer. What's really
important is the conceptual frame, though. In most studies of the costs
of testing, testing is an add-on. The whole view we have is that
assessment has to be part of the total system of professional activity
by educators.

What we have to ask is: What does it take, in terms of cost, to have
the kinds of professional development we need, to have the kinds of
networking of teachers who support each other and develop standards and
criteria? What kind of time off? This is probably the single greatest
cost for this kind of a system; what kind of time from teachers will it
take? And how much of that time is needed to be a good teacher? And we
ask then what additional cost is needed to provide a test score for the
public. We believe it will be very, very little over and above the cost
of the good education that we're aiming for.

Will there be new costs, a need for new resources for this? Yes. The
best estimate would be four, six, eight weeks of paid professional time
each year for teachers--that is, not with children, but rather
developing themselves professionally, working on curriculum, working on
the kind of preparation they need. In China and Japan, teachers spend
up to half of their paid professional time not facing children, but
developing their lessons, developing their material, and developing
themselves.

So, the question about the cost of assessment simply cannot be asked
in the traditional terms about what does it cost to administer and
score a test. It's the wrong way to view it.

Lauren B. Resnick, the director of the Learning Research and
Development Center at the University of Pittsburgh, is the co-director
of the New Standards Project, perhaps the most prominent effort under
way to develop a national system of assessments tied to high standards
for student performance.
One of the nation's leading cognitive psychologists, Ms. Resnick has
long argued that there is a mismatch between school practices and the
way people learn.
In place of the low-level, atomized skills emphasized in
schools--and on most standardized tests--she has argued, schools should
focus on a "thinking curriculum,'' which would emphasize students'
abilities to reason and solve complex, real-world problems.

Vol. 11, Issue 39, Pages s5, s6

Published in Print: June 17, 1992, as By All Measures: The 'Most Promising Way' of Getting the Education We Want

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.