Measure What Matters: What School Quality Means to School Stakeholders

Much of the public conversation about school quality rests on a reasonable assumption, namely that we can reliably assess school quality by the measures currently in use. This assumption is false.

Although not the sole measure, standardized test scores make up the bulk of school quality assessments. Chief among their shortcomings is narrowness. Standardized tests are blunt instruments that fail to reflect the full measure of what people most care about. As Harvard professor Dan Koretz, who studies educational measurement, points out, “Even a very good test measures only a modest proportion of what we value.” Left ignored by tests are things like the richness of the curriculum, the variety of extracurricular activities, access to support services, and students’ sense of safety. Finally, standardized test scores are strongly correlated with students’ demographic characteristics, which means they tend to tell us more about the number of economically advantaged students in any particular school than what they are learning.

Given these shortcomings, it is not too much of a stretch to say that we haven’t the foggiest idea how well schools are really doing. To help build a more comprehensive system for measuring school quality that goes beyond test scores, the Massachusetts Consortium for Innovative Educational Assessment (MCIEA) asked people who had a close-up view of what happens day-to-day in schools – teachers, principals, district administrators, family members, and students – what made a good school.

During focus groups with more than 250 participants from MCIEA school districts, the failures of the current system were readily apparent. The varied perspectives and in-depth stories we heard about what made schools work affirmed an essential truth shaping MCIEA’s school quality measures work: school quality is more complex and human than any test score or algorithm can capture. In response, we sought to design a school quality framework that better reflected the stories stakeholders told themselves about their schools.

One aspect of school quality continuously raised during focus groups was the relational dimension of education. This included teacher-student relationships, which researchers have consistently found have positive effects on students’ academic and social outcomes as well as teachers’ job satisfaction and engagement. One mother from Revere put it this way, “Kids look back on their time in school, and they’re always looking for that teacher that had that special relationship.” About her own children, she added, “The relationships they made with certain teachers carried into their adulthood, they’re still meaningful.” A high school student in Winchester similarly cited his strong connection with teachers as a primary motivation for coming to school: it’s “one of the things that makes you want to come to class,” he explained. Finally, a principal in Lowell asked, “How do we as adults in the building make connections with these kids so that there’s a touchstone every day – that he’s able to, she’s able to really touch base with someone who cares about them?”

In addition, focus groups helped highlight aspects of school quality that we had not considered. Among these was teachers’ professional community—the extent to which teachers felt connected to their school’s mission and to each other, which can boost collaboration and contribute to higher retention rates. Several teachers acknowledged the positive contributions of shared planning time and relationships with their colleagues, but the most vociferous voices in favor of adding a professional culture measure came from principals and families. A principal in Revere, for instance, observed that student-teacher relationships were vital, but just as vital were “staff or adult-adult relationships.” A parent in Attleboro added, “Being able to bounce ideas off of each other and being a team as opposed to being by yourself, I think, makes the teachers stronger overall and like their job more.”

Another measure that many stakeholders urged be included was cultural responsiveness. Several parents at a focus group in Somerville for non-English-speaking families agreed on the need for more inclusive and culturally responsive curriculum. In Lowell, one parent said that teachers’ ability to adapt curricula to be more inclusive of students from diverse backgrounds was critical toward making students feel comfortable and welcomed at school. A teacher in Winchester added that a culturally responsive curriculum was one that was “reflective or mindful of students of various backgrounds, that they see themselves – that they can see themselves in the literature they read and in the decorations in the classrooms.”

After the last focus group concluded, we synthesized themes and incorporated them into a comprehensive framework for measuring school quality. Rather than leaning on standardized tests plus a handful of other measures, we included 34 unique measures – a combination of performance assessments, administrative data, and student and teacher surveys.

Asking people what they value in schools is not, on its face, a radical act. But given how seldom parents and teachers and students are involved in decisions about what gets measured, it takes on added urgency. Because schools are being judged—sometimes harshly—by what gets measured, we need to make sure that we are measuring what people truly value and the only way to do that was to ask them.

Featured

The spark generated by performance assessments is catching at every grade level at Lincoln Elementary. If you want to see the true power of innovative assessments, simply talk to the folks who are in the thick of it, our students and teachers.

Educators, parents, students, and community members are eager to create a new education landscape in which the primary focus is on creating engaged, collaborative learners, readers, writers, problem solvers, and creators.

Also By This Blogger

Beyond Standardized Tests

Want to learn more about the work MCIEA is doing in the field? Our latest white paper features a case study with an MCIEA school and a behind-the-scenes look at the development of MCIEA's School Quality Measures.