Are Vermont students ready for the next grade?

Math expert and school consultant Jim Callahan of Middlebury: “When it comes to educational testing, words can be very tricky. You find terms such as ‘proficient’ and ‘proficient with distinction.’” He notes that ‘proficient’ sounds good to parents, but it’s deceptive; such a score meant the student passed only 40 percent of the test.

MIDDLEBURY | Two critics of how the State of Vermont reports its 2017 results of the Smarter Balanced Assessment Consortium student tests cite murky terminology, and difficult state-to-state comparisons, when it comes to explaining the data to parents and other shareholders.

John McClaughry of Vermont’s Ethan Allen Institute think tank, and instructional-materials author Jim Callahan—who owns Callahan Associates, his Middlebury-based education consulting and mathematics tutoring firm—claim that the Vermont Agency of Education is all too willing to emphasize that it’s too difficult to compare student proficiency levels of one state with another, for a number of so-called complicated reasons.

“Vermont Digger has done us another service by publishing a (recent online) report entitled ‘Making the Grade?,” according to McClaughry. “It presents the results of the tests used by Vermont and 14 other states to determine public school student proficiency in English and math in third, eighth and 11th grades.”

McClaughry said that while the Agency of Education, “goes to great pains to emphasize that it’s not possible to compare the student proficiency levels of one state with another... don’t leap to the unwarranted conclusion that public education is getting better results in state A over state B.”

McClaughry said that the key finding of the Digger report is that the average percentage of students achieving proficiency in Vermont this year was 48.4 percent, down from 50.92 percent last year.

Jim Callahan, the former principal of Middlebury’s Mary Hogan Elementary School and past member of the Middlebury School Board, has been railing against state testing results for years.

“When it comes to educational testing, words can be very tricky,” Callahan said. “You find terms such as ‘proficient’ and ‘proficient with distinction’. This started under No Child Left Behind.“

According to Callahan, the old test had cut off achievement scores with the lowest being termed ‘significantly below proficient.’

Despite a new test prepared by Los Angeles-based Smarter Balanced, Callahan is still troubled by the scoring criteria which declares a student either proficient or otherwise.

“So if a student fell into the ‘significantly below proficient’ level, it meant the he/she passed only 20 percent of the test material,” he said. “This suggests to me that the student may have guessed at most of the answers.”

Callahan noted that the test score level of ‘proficient’ sounds good enough to parents, but it’s deceptive; such a score meant the student passed only 40 percent of the test.

“Then we have the ‘proficient with distinction’ test score. What if all that a ‘proficient with distinction’ commercial airline pilot had to do was land at the correct airport 75 percent of the time?,” according to Callahan. “That’s exactly how Vermont grades student test takers score: A highly proficient with distinction) score means you got 75 percent of the test answers right. And this doesn’t even tell us what the student did to get to get to the answers. “

Today, the Common Core-linked achievement test levels are defined by what are called achievement level descriptors, the specifications for what knowledge and skills students display at each level (levels 1-4). But because each Smarter Balanced test member state, like Vermont, refer to them in different ways, such as “novice, developing, proficient, and advanced” it’s hard to make easy comparisons among the states.

According to the Smarter Balance website, “Students performing at levels 3 and 4 are considered on track to demonstrating the knowledge and skills necessary for college and career readiness. These achievement level descriptors were written by teachers and college faculty.”

Regarding McClaughry’s interpretation of the 2017 test data, “Half of our public school students finishing their respective school years are not ready to advance to the next grade. That’s measured by Vermont’s own proficiency standards, which may be too high, or too low; who’s to say?”

In the Digger investigation, reporter Tiffany Danitz-Pache wrote: “In almost every grade tested, in English and math, the percentage of (Vermont) students scoring ‘proficient’ or above was lower than in 2016. Low-income students on average couldn’t reach the state average score in either subject in any grade. And while the gap between low-income and wealthier peers is closing, the shift is due to better-off peers losing ground.”

McClaughry concludes that when it comes to achievement testing in Vermont, it’s much like entertainer Bill Murray’s starring role in the 1993 movie, “Groundhog Day”.

“We have gone through half a dozen assessment regimes/fads in my 50 years,” he said. “Not long after our kids don’t do so well, the current assessment is quietly shelved and an exciting new model is rolled out (remember Camp Portfolio?)... For a billion and a half dollars every year, it seems to me that we ought to be getting better than 50 percent proficiency.”

Note: In an upcoming article, we will include responding comments about testing in Vermont from officials of the Agency of Education, local school districts, and parents.