Florida, which bought and uses Utah’s SAGE/AIR test, has taken the phenomenally reasonable step of assessing its assessment: testing the standardized test– something that Utah has not done.

Florida hired Alpine Testing and EdCount to assess its (and Utah’s) assessment instrument –to see if the SAGE measures what it claims to measure. The simple question was: Is the test valid?

The answer that came back was “NO.” The independent company, Alpine Testing and EdCount, who testified at length to the Florida legislature, said that SAGE is not measuring what it claims to measure. (See that legislative testimony here.)

Now, two members of Utah’s largest school district (Alpine) have published a letter summarizing Florida’s findings on SAGE. Brian Halladay and Wendy Hart wrote:

“What Alpine Testing said in their comments to Florida is astounding. I have outlined some key points from the video:

At 44:50- Many items found in the test didn’t align with the standard that was being tested.

At 47:70: Test scores should only be used at an aggregate level.

At 48:15 – They recommend AGAINST using test scores for individual student decisions.

At 1:01:00 – They admit that “test scores should not be used as a sole determinant in decisions such as the prevention of advancement to the next grade, graduation eligibility, or placement in a remedial course.”

At 1:20:00 – “There is data than can be looked at that shows that the use of these test scores would not be appropriate”.

Alpine Testing was the only company that applied to perform the validity study for Florida. Once awarded the contract, they teamed with EdCount, the founder of which had previously worked for AIR.

So, what we have is a questionably independent group stating that this test should not be used for individual students, but it’s ok for the aggregate data to be used for schools and teacher evaluations. If this sounds absurd, it’s because it is. If it’s been shown that this test isn’t good for students, why would we be comfortable using it for the grading or funding of our schools and teachers? The sum of individual bad data can’t give us good data. Nor should we expect it to.

What more evidence is needed by our State Board, Legislature or Governor to determine that our students shouldn’t be taking the SAGE test? This test is a failure. How much longer will our children and our state (and numerous other states) spend countless time and resources in support of a failed test, or teaching to a failed test?…” (Read the whole letter here.)

Why is this so important?

Any test– a pregnancy test, a drug test, a breathalizer test– should probably actually measure what it claims to measure. People should be able to solidly trust a test that’s used as a foundation for labeling, rewarding and punishing students, teachers and schools.

If there’s no validity test, SAGE is nothing more than a gamble with children’s, teacher’s, and taxpayer’s time, money and futures. Without validity, we’ve just conscripted every public school student in the state to be unpaid, uninformed, academic and psychological lab rats.)

Fact: Utah stubbornly refused to do a validity test on SAGE, despite pleading, prodding, and even a $100,000 reward offer for proof of validity testing –yet, as it turns out, that’s okay now. Since Florida uses Utah’s SAGE test, Florida’s research on SAGE directly, unquestionably, reflects on Utah’s test. So we finally have a Utah validity test. And SAGE failed its test.

Thanks so much for this news. As I understand it computer adaptive tests do not measure student achievement. If it is true that test questions change depending on the answers given by students, then you can’t compare them. If a student can’t skip a question that he or she doesn’t know or wants to answer, but the test won’t let them do that, then the test is only measuring the “grit” or determination of the student. We will never know exactly how the student answered a question; for example, if the student used the number line to solve a problem and thought the answer was a fraction and the machine test only accepts a whole number answer, how do we know how the student responded if the computer moves the response to a whole number. It’s a mystery to me why anyone would want to use or accept these tests? Am I missing something? Melanie Thomas

The following time I read a weblog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my choice to learn, however I actually thought youd have one thing fascinating to say. All I hear is a bunch of whining about one thing that you may repair when you werent too busy on the lookout for attention.