State to State, Varied Ideas of 'Proficient'

By SUSAN SAULNY

Published: January 19, 2005

Judged solely by recent statewide tests, fourth graders in Mississippi and Colorado would appear to be the best young readers in the nation. In both states, 87 percent of fourth graders passed their exams.

But Mississippi came in dead last among the 50 states when fourth-grade reading was examined using a different standard, a newly mandated but decades-old test called the National Assessment of Educational Progress, or N.A.E.P. On that test, only 18 percent of Mississippi's fourth graders achieved proficiency.

Colorado's proficiency rate fell to 37 percent on the national test, but that score was high enough to rank fifth in the nation.

Such comparisons of performance on state tests versus national tests have never been possible before on a nationwide basis. The N.A.E.P., known as the nation's report card, used to be voluntary for states. In 2003, it became mandatory. The comparisons suggest how widely the definition of ''proficient'' varies from state to state, as each administers its own exams and sets its own performance standards.

And those standards matter more today than ever, because they factor into the federal education law, No Child Left Behind, which requires that all students reach proficiency on state reading and math tests by 2014. States are also judged on yearly progress toward that goal, and harsh penalties, including the loss of federal aid, await those that fail to bring all students to a proficient level.

''No Child Left Behind leaves a fairly crucial decision'' about defining proficiency up to the states, said Ronald A. Skinner, research director of Education Week, which published the state and national scores side by side recently in an annual survey of schools. ''Certainly those states that have set the bar lower will have an easier time meeting the mark and avoiding federal sanctions for their schools. It's going to be tough on states that have put tough standards on their students.''

Researchers and educators say the new data will make it possible to address questions they could not answer before.

''When you compare yourselves using N.A.E.P., you're able to compare yourselves to a much more expansive and comprehensive national base,'' said Douglas E. Wood, executive director of the National Academy for Excellent Teaching at Teachers College at Columbia University. ''It seems to me that offers us additional information by which to make policy decisions.''

In Mississippi, Kristopher J. Kaase, the director of the Department of Education's office of student assessment, said the state had traditionally used its own definition of the word proficient.

''We call it solid academic performance required for success at the next grade level,'' Mr. Kaase said. ''Step away from that for a moment. Who's ready to move on to the next grade level? At least a C student. I don't think anyone would have any qualms about that. Would they be proficient according to N.A.E.P.? Probably not.''

The smallest disparities between results on the state and national tests were found in a variety of states across the country, including Massachusetts, Maine, Wyoming, South Carolina, Vermont and Missouri.

The largest disparities were in the South. In Alabama, for instance, 72 percent of fourth graders were proficient on the state's math test, but only 19 percent passed N.A.E.P. at the proficient level.

In New York, the gaps between state and N.A.E.P. scores were much smaller, but the state did not fare as well as Connecticut or New Jersey on the national test. Thirty-five percent of eighth graders in New York were proficient on the N.A.E.P. reading test, compared with 45 on the state test. Thirty-three percent of New York fourth graders were proficient on N.A.E.P.'s math test, while 78 percent passed the state math test.

Mr. Kaase said that No Child Left Behind has made Mississippi wary about raising standards.

''What is already a challenging goal -- reaching 100 percent proficient by 2014 -- you can make it much more challenging or nearly impossible, depending on what you do,'' he said. ''It becomes a delicate balance. But we do feel we need to continue to press. We're not satisfied.''

Colorado's definition of proficient, a state official said, was changed to comply with No Child Left Behind, which requires that results be reported in three categories. The state had for years reported its test results in four categories: unsatisfactory, partially proficient, proficient and advanced.

To meet the new requirements, Colorado grouped its partially proficient students with the proficient.

''We had a dilemma,'' said William J. Maloney, the commissioner of education. ''We would have had to throw our whole system in the Dumpster just to accommodate the N.C.L.B. So we said: 'Here is Colorado. For the purposes of the feds, we combined proficient and partially proficient.'''

Unlike Mississippi and Colorado, Missouri adheres to a strict definition of proficient that aligns closely with the N.A.E.P., or surpasses it. In 2003, eighth graders in Missouri did better on the N.A.E.P. reading test than on the state's own exam.

''We've tried to look at it as 'How can we best serve our kids?' instead of trying to play some numbers game against federal law,'' said Bert Shulte, the deputy commissioner of education in Missouri.

But Mr. Shulte realizes the long-term risks of Missouri's decision. ''It makes it harder for us to achieve a federal numerical goal,'' he said. ''But in terms of what it says to students when they achieve proficiency, it is a better-grounded message.''

Some educators and policy makers say that rather than focusing on a state's performance on a single test, the goal should be improvement over time.

''On the one hand, you can say, if there's a huge disparity, if a state is telling parents that 80 percent of students are proficient but on N.A.E.P it's 20 percent, they're lying,'' said Kati Haycock, the director of the Education Trust, a group that advocates standards. ''While that may at some level be true, in order to make progress in education you have to have near-term goals that are achievable. So if you're a state with really low achievement and you say, 'I've got to set my state's bar where N.A.E.P. is, and make it under N.C.L.B.'s timeframe,' I think educators would throw up their hands and say, 'We can't get that far.'''

Ms. Haycock cited North Carolina and Texas as examples of states that have shown the biggest gains on the N.A.E.P. over the last 10 years while having low-level state assessments.

Mississippi fits into that picture as well. In 2003, for instance, the number of fourth graders scoring at or above the proficient level on the N.A.E.P. math exam more than doubled to 17 percent, from 8 percent in 2000, while 74 percent of fourth graders were deemed at or above the proficient level on the state test.

''And so, where a mismatch as large as you see in some states may be worrisome, considering near-term targets, I think the evidence would not suggest that's a bad thing,'' Ms. Haycock said.

Mark Musick, the president of the Southern Regional Education Board, an organization that advises 16 states on education policy, said he has been arguing for years that ''the states should look to N.A.E.P. and any other information, take it and see if the results are telling them basically the same thing.''

Mr. Musick, a former chairman of N.A.E.P.'s governing board, added, ''If there's some wild difference, then they should try to figure out why that difference exists.''

Chart: ''Telling Disparities in Selected States''
The percentage of students scoring at or above proficiency in state tests often far surpasses the percentage who achieved proficiency on the National Assessment of Educational Progress, or N.A.E.P.