Call for ‘Scientifically Based’ Programs Debated

The question, posed at a recent conference here, was this: Is the
federal government’s call for "scientifically based research" in
education a mandate or just strong advice?

Nearly 26 months after President Bush signed the federal No Child
Left Behind Act, that question still looms.

Although the law is known best for its bold testing and accountability
requirements, it has helped put the phrase "scientifically based
research" on the lips of educational decisionmakers from coast to
coast. It calls on educators to use research to guide a wide range of
school practices, including reading instruction, professional
development, technical assistance, and anti-drug programs.

Part of the problem is that the legislation itself contains two
different definitions of the phrase—one for Reading First,
President Bush’s flagship reading initiative, and one to cover
the rest of the law’s provisions.

Since the law’s passage in 2001, the U.S. Department of
Education has crafted regulations spelling out what it means by
scientifically based research with regard to the reading program.

But no such regulations exist for the rest of the law—mostly
because federal policymakers view the research base as weaker in those
areas. As a result, even some of the law’s primary architects and
enforcers are reluctant to call the phrase a mandate.

"It certainly is not a druthers situation. It is the law," said
Robert Sweet, who helped define "scientifically based research" in the
law as a staff member on the House Committee on Education and the
Workforce. "In practice, is everyone going to be held to a strict
scientifically based research definition?" he continued. "Probably
not."

No Inspectors

The question about scientifically based research came up at a March
11 conference on the topic organized by three Washington-based
organizations: the congressionally chartered National Academies of
Science, the National Education Knowledge Industry Association, a
research trade group, and the Progressive Policy Institute, a centrist
think tank.

At the meeting, both Mr. Sweet and Grover J. "Russ" Whitehurst, the
director of the Education Department’s Institute of Education
Sciences, said no "scientifically based research police" would be
coming around inspecting schools. In due time, though, they hope
educators will come to rely on such evidence.

States, on the other hand, are taking the federal law at its word,
according to Ted Sanders, the president of the Education Commission of
the States, a Denver-based research group for state policymakers. "I
don’t think there’s any state in the country that looks at
this as advisory," he told conference-goers.

In fact, in a survey published this month by the Center on Education
Policy, a national advocacy group based here, 64 percent of the
districts polled said they require schools participating in the federal
Title I program for disadvantaged students to select programs from a
list of curricula or instructional programs grounded in scientifically
based research.

Washington state, for example, last year turned down half the
experts who applied to present at state-sponsored
professional-development meetings for lack of scientific evidence,
according to Dawn Billings, as assistant superintendent in that
state’s education department.

"We have to do things differently," she said last week. "We
can’t go higgledy-piggledy doing this reform and that. We are
taking this very seriously."

Strict Adherence?

Even so, Jack Jennings, the center’s director, thinks the
numbers his survey turned up could be inflated.

"I think there’s a difference in people’s minds about
whether they’re trying to use scientifically based research in
improving education and whether they’re strictly following the
requirements of federal law," he said in an interview last week.

Both of the definitions of scientifically based research under the
No Child Left Behind law call for studies that apply "rigorous,
systematic, and objective" procedures and have been subjected to peer
review.

But the definition that applies specifically to all programs outside
of reading goes a few steps further. It expresses a clear preference
for either randomized experiments or quasi-experiments, in which the
group that receives the experimental treatment can be compared with
another group that does not.

The definition also says studies should be detailed enough so that
their findings can be replicated.

Though the Reading First definition is the more flexible of the two,
states appear to be interpreting it very narrowly, partly because the
Education Department sent many states’ program applications back
for numerous revisions. ("Reading Programs Bear Similarities
Across the States," Feb. 4, 2004.)

Whether from a desire to appease federal reviewers or confusion over
how to meet the new requirements, some states modeled their programs on
those of the successful applicants that went before them.

In the process, some reading programs that are widely considered to
have solid research track records were sidelined.

Success for All, a school improvement program established by
researchers at Johns Hopkins University in Baltimore, is a case in
point. Rated in 1999 by the American Institutes of Research as one of a
handful of programs with "positive" evidence of effectiveness, the
program got left off some of the earliest approved-program lists that
states put together for Reading First.

"Daily, we’re getting reports from one state or another where
somebody is being told Success for All is not a legitimate use of
Reading First funds," Robert E. Slavin, the program’s founder,
said in an interview. "There’s an undercurrent that, if you want
to be safe, you’re better go with one of the big [commercial]
series."

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.