Making Reliability Arguments in Classrooms

Online Submission, Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Francisco, CA)

Reliability methodology needs to evolve as validity has done into an argument supported by theory and empirical evidence. Nowhere is the inadequacy of current methods more visible than in classroom assessment. Reliability arguments would also permit additional methodologies for evidencing reliability in classrooms. It would liberalize methodology beyond reliability coefficients and agreement indices enabling and encouraging more creative yet disciplined approaches to supporting reliability. But how does a teacher actually make a reliability argument and, more specifically, support it with evidence? Where is reliability--stability and consistency--evident in the organic and all-too-human day-to-day operations of a classroom? This paper outlines reliability arguments and shows examples of them with a collaborative project, in assigning a homework grade on a report card, and in discussing behavioral issues with a parent in a parent-teacher conference. Scoring Rubrics for the Collaborative Project is appended. (Contains 1 table and 1 footnote.)