A Living Laboratory

The law requires districts to provide teachers with at least four
days of in-service training.

Such positive findings, which have come out only in the past year,
still may not be enough to sway teachers, who report in survey after
survey that they do not trust the test results. Many believe students
are scoring higher because they are becoming more familiar with the
tests.

Emphasis on Writing

There is less doubt, on the other hand, that the tests have changed
the mix of activities that take place in the classroom.

Ms. Bridge, of North Central College, studied instructional
practices at four Kentucky elementary schools in 1982. She returned to
the same schools in 1995, five years into the reforms, to see whether
teaching had changed.

She found that in the grades she studied--1st, 3rd, 4th, and
5th--teachers were doing 1.2 to two times as much writing with their
students as they did before the reforms. Students were writing
expository essays, newspaper articles, stories, paragraphs, and essay
responses of the type called for in the state testing system.

"If you think about doubling the amount of time that teachers spend
[on] writing, I think that's pretty significant," Ms. Bridge said
recently. A handful of other studies echo her findings.

But teachers also complain that all the writing, rewriting, and
conferring over writing involved in the new approaches eats up too much
classroom time. "Teachers in Kentucky felt they had to give up
important material," said Mr. Koretz, who surveyed 500 Kentucky
teachers and principals during the 1994-95 school year.

What they have sacrificed, teachers typically report in surveys, is
instruction in basic computational and language skills.

To test whether students were indeed losing out on those skills, the
state hired the Human Resources Research Organization, a Radcliff,
Ky.-based group, to compare examples of student writing from 1993 and
1996. The researchers analyzed the samples for spelling,
capitalization, punctuation, and subject-verb agreement.

"The long story is that 4th grade girls' performance essentially
remained flat, and the boys improved," said Gene Hoffman, the manager
of the group's center for learning, evaluation, and assessment
research.

Success With Preschool

The 1990 law called upon districts to set up preschool programs for
at-risk 4-year-olds and for children ages 3 and 4 with
disabilities.

A year later, researchers began tracking groups of children who
entered the program and comparing them with other youngsters who were
eligible for preschool but whose parents decided against enrolling
them. The sample now includes about 3,000 children, the oldest of whom
are in 5th grade, according to Mary Louise Hemmerter, an associate
education professor at the University of Kentucky who is working on
that project.

"What we are finding is that these children make progress," she
said. "In kindergarten, teachers rate them as prepared as children from
higher-income families and more prepared than children who could've
gone through the preschool programs but did not."

Over time, the study also found, the former preschool pupils kept
pace with their better-off peers. And those students' grades did not
drop off in 3rd grade--the point at which studies of other preschool
programs suggest that achievement gains made in the early years start
to fade.

Training Focus Studied

In the area of professional development for teachers, another
bulwark of the reform program, the research evidence is more mixed. The
law requires districts to submit their plans for professional
development to the state and to provide teachers with at least four
days of in-service training. Schools also received extra money to pay
for the added training.

But Thomas Corcoran, a co-director of the Consortium for Policy
Research in Education, a national organization based at the University
of Pennsylvania, said schools have not made the wisest use of those
opportunities. He and two colleagues studied 20 schools noted for doing
"interesting things" in professional development. They reviewed those
schools' plans and talked to heads of their professional-development
committees.

"The focus in most places was on the short term," Mr. Corcoran said.
"It was, 'What can we do with the money we have and the four days we
have to jack up our KIRIS scores?'" As a result, activities tended to
focus on pedagogical techniques, such as cooperative learning, and new
assessment strategies. Little attention was paid to deepening teachers'
understanding of the subjects they teach.

On the other hand, he added, "the good news is that there's more
professional-development activity and more of it is schoolwide, and
there's really an effort to get some practical payoff out of it."

"Even though the definitive word on KERA's success or failure is
yet to come, no one in Kentucky has advocated throwing out the
reforms."

The reform package also called upon schools to establish extended
programs to offer help to struggling students before and after school
and during the summer. Schools with a high proportion of poor students
were also required to set up resource centers where families could
receive help obtaining eyeglasses or medical referrals for their
children. And studies on those programs suggest they are up and running
and doing their jobs.

Broader Research Needed

While such research on individual provisions of the reform law is
useful, Ms. Lindle of the University of Kentucky says, overall the
studies on its impact have been mostly small, fragmented, and not
nearly as numerous as researchers and state officials might have hoped.
"Here we've got a massive reform system and no real window into it,"
she said.

Research gained a higher profile after Wilmer S. Cody became the
state's second appointed superintendent in 1995. He named William White
last year to be the first research manager for the education department
since it was re-created in 1990. The department is also devising a
peer-review process to raise the quality of the studies it pays
for.

Mr. White says the time has come to shift the focus away from
studies that look just at the reform program's "inputs."

"We are at the point that most of the pieces are in place, and we
need to devote more time to evaluating the effectiveness of programs,"
he said. "For example, we know a lot about what kinds of technology we
have in place and where, but we don't know a lot about how technology
is actually being used in the classroom by our teachers." The
department is in the process of commissioning studies to answer that
question.

In a similar vein, the Kentucky Institute for Education Research,
which Mr. Pankratz heads, is re-examining its mission. The
organization, with support from the Annie E. Casey Foundation, has
bankrolled much of the research on the reform program to date. The
institute's funding is scheduled to run out before the close of the
century, prompting its leaders to review what role it will play in
future research.

Even though the definitive word on KERA's success or failure is yet
to come, Mr. Pankratz says, no one in Kentucky has advocated throwing
out the reforms.

"I think we have been effective in trying to convince legislators
and others to improve what we have rather than starting over," he said.
"Whether we're going to be fortunate enough to learn from our mistakes
and successes--that's not been answered."

PHOTO: Patricia Kannapel of the Appalachia
Educational Laboratory is a co-director of a research team that since
1991 has conducted fieldwork on the effects of Kentucky's 1990 reform
law on four districts.
--Nick Romanenko

Roger Pankratz, the director of the Kentucky
Institute for Education Research, says the longevity of the reforms has
allowed time for study.
--Lonnie Harp

Web Only

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.