How to Measure Student Proficiency? States Disagree on Setting Standards and Tests

By FORD FESSENDEN

Published: December 31, 2003

The community around South Charlotte Middle School is one of the richest in North Carolina, and the school boasts the kind of test scores that seem to go hand in hand with wealth. Last year, more than 95 percent of its students passed both the state reading and mathematics tests.

A few miles away in a similarly wealthy community, the students at Fort Mill Middle School cannot make the same claim. More than half failed the state mathematics test, and three-quarters failed the reading test.

The difference? Fort Mill Middle School is in South Carolina.

Two recent studies show that such anomalies are widespread, as states have set widely different standards for measuring students' progress under the federal education law known as No Child Left Behind. Three-quarters of children across the country would fail South Carolina's tough fifth-grade test, one study shows, while seven out of eight would ace the third-grade tests in Colorado and Texas.

The two studies, one by a nonprofit Oregon testing company and the other by a Washington interest group, take different routes to reach a similar conclusion: Across the country, there is no agreement on how much students need to know to be considered proficient.

''It means parents and students are getting very different signals about what it means to be well educated, what it means to be prepared when you leave school,'' said Michael Cohen, president of Achieve Inc., a nonprofit school reform group in Washington that released a study of state standards in November.

The divergent standards also have ramifications under the federal education law, passed in 2001. Schools deemed failures eventually face stern consequences, including loss of students and reorganization. And in some states with high standards there could be lots of failing schools. In other states with low standards, schools with equally poor performance could be left alone.

''States with low standards will have relatively few schools that experience the consequential aspects of the law,'' said Allan Olson, president of Northwest Evaluation Association in Lake Oswego, Ore., and co-author of the other study. ''With standards as high as they are in some other states, it's likely that most of the schools will be under sanction.''

The studies come as states begin to chafe at the demands and inconsistencies of the education law. Last month, Public Agenda, a nonprofit public opinion research group, released results of a survey that said that school principals and superintendents were deeply suspicious of the law and that most think it will not work without changes.

Some experts agree with that view.

''If it's not changed, it will collapse of its own weight,'' said Robert L. Linn, co-director of the Center for Research on Evaluation, Standards and Student Testing at the University of Colorado. He added, ''I think a lot of people already realize that, but they're waiting for a politically viable time to make the adjustments.''

But Kati Haycock of the Education Trust, an education standards advocacy group in Washington, said the grousing amounted to inevitable growing pains for the federal accountability system. ''To me, getting a lot of pushback is actually a good sign, but it's important not to overreact to the pushback,'' she said.

''It may be that after another year or two's worth of data we may have to change something,'' she added. ''But I don't think it's clear now, especially because states have the ability to change their standards.''

Officials at the United States Department of Education agreed. ''The states are free to set their own standards to meet the needs of children in their states,'' said Ron Tomalis, acting assistant secretary in the Office of Elementary and Secondary Education. He added: ''Communities may say we want to have higher standards. Communities may say we've set it way too high, we may want to set it down lower toward the norm.''

The Northwest Evaluation study was based on scores of students in 14 states who took both the state proficiency test and one of the organization's own tests. The group claims a high degree of reliability in its estimates, even though it may give its tests in just one school district, because they are based on scores of hundreds of students taking both tests.

Colorado's reading test was consistently the least demanding in most grades in which it was given, with a passing score that corresponded to a national ranking between the 9th and 18th percentile. South Carolina and Wyoming had passing scores in the 70th percentile and higher in most grades.

The study by Achieve, a nonprofit group that promotes high education standards, compared the number of students the states had declared proficient under their No Child Left Behind testing structure with the number at the ''proficient'' level on the National Assessment of Educational Progress, a test given in all 50 states. It showed Louisiana, South Carolina and Wyoming setting some of the highest bars, with Texas, North Carolina and Mississippi setting some of the lowest.

Texas, whose apparent progress on a state test in the 1990's helped pave the way for No Child Left Behind, was near the bottom in both studies, even though it started using what it said was a more difficult test last year.

In South Carolina, which adopted a standard for a statewide accountability system three years before the federal requirement was passed, officials feel caught in a bind: Do they lower proficiency standards, or risk stigmatizing most of their schools and opening them up to sanctions?

''The ramifications are much more pronounced in a state that has high standards,'' said Inez Tenenbaum, the state superintendent of education.

More than 75 percent of the state's schools failed to make the progress required by the federal education law this year, far more than the 3 percent Alabama reported, or the 8 percent reported by Texas. One of them was Fort Mill Middle School.

''That hurts,'' said Marty McGinn, director of middle and and secondary education for Fort Mill School District No. 4. ''It has had a great impact on the faculty.''

Bob Ormseth, spokesman for the district, said: ''No one here at Fort Mill Middle is saying they're going to move their children. The attempts by the state and federal government to label the schools here have not diminished parents' belief in the performance they see, and their confidence in the schools.''

Ms. Tenenbaum said, ''We don't want to lower our standards,'' and added, ''We think everyone ought to have as high a standard as we do.'' But, she said, ''There ought to be some national clearinghouse so we can have a comparative measure.''

Chart: ''Disparities in Testing'' Proficiency in fourth-grade reading as determined by state and national testing, in selected states. Louisiana STATE TESTS: 14% NATIONAL TEST*: 20 South Carolina STATE TESTS: 31 NATIONAL TEST*: 26 Maine STATE TESTS: 49 NATIONAL TEST*: 36 Massachusetts STATE TESTS: 56 NATIONAL TEST*: 40 California STATE TESTS: 39 NATIONAL TEST*: 21 Connecticut STATE TESTS: 69 NATIONAL TEST*: 43 Colorado STATE TESTS: 63 NATIONAL TEST*: 37 Florida STATE TESTS: 60 NATIONAL TEST*: 32 Arkansas STATE TESTS: 61 NATIONAL TEST*: 28 Michigan STATE TESTS: 75 NATIONAL TEST*: 32 Vermont STATE TESTS: 81 NATIONAL TEST*: 37 Wisconsin STATE TESTS: 80 NATIONAL TEST*: 33 North Carolina STATE TESTS: 81 NATIONAL TEST*: 33 Texas STATE TESTS: 85 NATIONAL TEST*: 27 Mississippi STATE TESTS: 87 NATIONAL TEST*: 18 *National Assessment of Educational Progress (Source by Achieve Inc.)