Study Disputes Link Between Reforms, Rise in Test Scores

Washington--The recent rise in the scores of students on
standardized tests is not attributable to this decade's wave of school
reforms, a new study paid for by the Congress argues.

The upswing in standardized-test scores not only predates education
reforms of the 1980's, but "might well have continued in their
absence," states the report, prepared for the Senate Education, Arts
and Humanities Subcommittee by the Congressional Budget Office.

The report disputes common assumptions about why scores fell in the
1960's and began rising in the mid-1970's--such as that students
watched too much television, or that high-school graduation standards
declined.

And it contends that, in any event, test-score trends are an
inadequate yardstick for measuring reform, which it says hinges on a
far more complex range of factors than the tests measure.

A more complete portrait of achievement trends could be painted, it
suggests, by juxtaposing a wide array of data weighing academic,
demographic, and societal factors.

Daniel M. Koretz, the cbo's analyst for elementary and secondary
education, wrote the 104-page analysis, the second of two studies on
testing by the budget office.

"Many people have used trends in test scores and assumptions about
their causes not only to formulate new education policies, but also as
a basis for presuming their effectiveness," the report states.

But, the cbo asserts, "the results of this analysis suggest that the
effectiveness of the current wave of initiatives should not be presumed
on the basis of assumptions about what caused past trends."

Cause-and-Effect Gap

Public awareness that student achievement seemed to slip sharply in
the 1960's and 70's, lagging also in international comparisons, has
helped drive a wide range of reforms, including higher pay and stricter
certification standards for teachers to expanded testing and more
rigorous requirements for students.

While the cbo report does not disparage policies intended to counter
declining test scores and the factors presumed to have causedthem, it
cautions that "assuming greater consistency than actually exists"
between the trends and the initiatives can misdirect policymaking and
obscure the importance of other contributing factors.

The report says demographic and cultural shifts, rather than stiffer
graduation standards and other hallmarks of reform, have influenced
test-score trends.

But, it cautions, the mix of other factors that fueled achievement
fluctuations remains largely unknown.

The cbo analysis found, for example, that changes in the size of
students' families and in the ethnic mix of test takers made
"particularly substantial contributions" to testing trends. Still, the
report states, each of the two factors accounts for at most a fifth to
a quarter of the test-score decline.

The growing percentage of minority students--who, on average, score
lower on standardized tests--contributed to the decline and slowed the
subsequent rise, according to the report; the changes in family size
associated with the baby boom and bust, in the meantime, influenced
shifts in both directions.

Misconceptions Assessed

Factors that probably did not contribute significantly to the trends
include many that "have gained widespread credence as possible causes,"
states the report, "Educational Achievement: Explanations4and
Implications of Recent Trends."

The report--which expands on a 1986 cbo study that analyzed
achievement patterns measured by several standardized tests (See
Education Week, April 23, 1986)--says many observers, focusing mainly
on the Scholastic Aptitude Test, link dips in scores to weak policies
and relaxed requirements at the high-school level and attribute the
rise in scores to the success of school reforms.

But in looking at a broad range of elementary- and secondary-school
test data, the cbo found that the students whose test results triggered
the upward trend in scores entered school in the late 1960's--a fact
that links their performance to educational and social policies
predating those of the reform era.

Reform Movement Credited

The report notes that state graduation standards did not change
significantly between 1974 and 1979 and, thus, "appear not to have
contributed directly" to the downturn intest scores during that
period.

The cbo's findings cast doubt on claims by education officials,
including Secretary of Education William Bennett, that the rise in test
scores demonstrates the reform movement's success in recent years. Mr.
Bennett in 1985 linked gains in sat score results to raised academic
standards and declared the movement was holding its ground when the
scores leveled in 1986.

Bruce Carnes, the undersecretary for planning, budget, and
evaluation, said last week that the department has no "fundamental
problems" with the cbo's study. The department "never ascribed all of
test-score performance to a particular thing," but still sees education
reform as a key player, he said.

"We don't think it's voodoo or sunspots," said Mr. Carnes, who
cautioned against construing the study to mean that cultural and family
characteristics outweigh human will and determination to succeed.

While those who link score trends and education reforms often assume
a few key factors can be singled out and reversed to improve scores,
the cbo study suggests that "changes in many, diverse educational
factors might well be necessary to bring about increases in achievement
as pervasive and large as the decline of the 1960's and 70's."

The report emphasizes that, despite the upward test trend in recent
years, "the average level of perform8ance on some tests remains well
below what many educators would consider acceptable."

'Plausible' Factors Cited

Citing factors that could be viewed as "plausible" causes of
achievement trends, the report says a watering down of course content
in secondary schools may have contributed to the decline in test
scores. And it suggests that modest changes in the amount of homework
assigned high-school students may have influenced the decline and rise
of scores.

The report also states that the federal Chapter 1 program "could
have contributed modestly" to the appreciable academic gains registered
by black and Hispanic pupils relative to other students and that
desegregation efforts may have bolstered the gains of blacks.

Further, changes in students' use of alcohol and other drugs may
have figured in the decline and rise in scores, the report says, and it
notes that reduced exposure to lead in the environment "might have
contributed in small measure to the upturn."

But students' television-viewing habits cannot be linked to the
trends because they have not changed substantially during the past two
decades, the report states.

Without dismissing the relationship between teacher preparation and
student achievement, the report also discounts the theory that the
declining sat scores of prospective teachers depressed student
achievement, since the decline did not begin until 1972.

The report contends that test-score patterns in some cases can mask
or downplay the significance of education policies. For example,
unusually rapid demographic changes or successful campaigns to lower
the dropout rate may depress average scores "even if policies carried
out during that time are beneficial," the cbo says.

Some potentially powerful factors whose influence on trends in test
scores cannot be evaluated for lack of data, according to the report,
include student attitudes and motivation, schools' emphasis on writing,
and local graduation requirements.

The cbo adds that initiatives aimed primarily at high-school
students may underrate the importance of elementary schooling, where
critical skills are first taught.

Alternative Measures Urged

The report also says a growing tendency by teachers to "teach to the
test," or gear instruction toward boosting test scores, could skew the
significance of those scores. "Regardless of whether increased teaching
to the test is desirable, it is likely to make trends in test scores a
distorted proxy for achievement," it states.

The report urges educators to devise ways to address students'
deficiencies underscored by the test data in "higher order" skills and
to boost the performance of traditionally low-scoring groups. The
report mentions recent policy proposals aimed at broadening the
nation's portrait of student achievement, including the Education
Department's proposal to expand the National Assessment of
Educationel10lal Progress to allow state-by-state comparisons. (See
Education Week, March 25, 1987.) Mr. Carnes termed that project "one of
the most valuable tools" the Reagan Administration can contribute to
education policy.

The report's analysis "argues strongly against relying solely on a
single 'national achievement test"' for information about student
performance. It would be "more reliable and informative," it argues, to
use a number of tests.

"A comparison of several tests is often necessary to discern which
results are consistent enough to provide a sound basis for policy," it
states, "as evidenced by the several important instances in which the
National Assessment of Educational Progress has yielded conclusions
that are inconsistent with other data, and the wide variation in the
results shown by other tests."

Copies of the cbo report may be obtained at no cost by writing the
Publications Office, Congressional Budget Office, U.S. Congress,
Washington, D.C. 20515.

Web Only

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.