California at bottom in nationwide ranking of accountability systems; state board president disagrees

November 14, 2017

November 14, 2017

Another prominent education research and advocacy organization that disapproves of California’s approach to school accountability has ranked California’s new system at the bottom nationwide in a report released Tuesday.

The low score by the Thomas B. Fordham Institute reflects a core disagreement over how best to identify and work with schools needing help. California education leaders are unapologetic about the route they’ve chosen, and they say the Fordham analysis contains a key error.

Related

Like Bellwether Education Partners, which harshly criticized the state’s approach in an August analysis, Washington, D.C.- and Ohio-based Fordham gives high grades to states that will rank schools with an A-F letter grade or a similar method that’s understandable at a glance. States will use rankings to select the lowest-performing schools, as required by the federal Every Student Succeeds Act.

California’s color-coded school dashboard does not give a summary school ranking. Each measure of performance, whether test scores, graduation rates or student suspension rates, gets a separate color rating. Gov. Jerry Brown and the State Board of Education say that this approach focuses attention on specific areas that need work. While this is more complex — and, some critics say, confusing — advocates say it is more helpful in diagnosing problems.

Fordham also rated California poorly for not giving more weight to how much students’ test results improve year over year. And it criticized its current methodology to measure schools’ growth in scores.

In a statement, State Board of Education President Michael Kirst said Fordham’s ranking judged California’s system narrowly and failed to appreciate “the totality of California’s efforts to improve outcomes for all students.”

“This report arbitrarily picks out only 3 components of a large multi-component plan to rate states,” Kirst wrote in an email. “The California School Dashboard provides communities with unprecedented access to meaningful information about schools,” he wrote. “The Dashboard’s Equity Report makes achievement gaps impossible to ignore.” The equity report breaks down student performance by demography, race and ethnicity and other factors, highlighting disparities.

But in its report, Fordham praised the 34 states that used either A-F ratings or “user-friendly numerical systems.”

“Easy-to-understand labels, such as A–F letter grades, provide clear signals to parents, citizens, and educators about the quality of a school and can nudge systems toward improvement. ‘Dashboards’ — which present lots of data points but no bottom line — are great complements to labels, especially when teams sit down to determine how to take a school to the next level, but they are no replacement,” wrote authors Michael Petrilli, the institute’s president, and Brandon Wright, the institute’s editorial director.

California’s color scheme is a work in progress. Early next year, the state board will adopt a methodology for applying color ratings for multiple indicators on the dashboard to identify lowest-performing schools. So far, the board and the California Department of Education have found this to be harder to do than they anticipated.

The report examined states’ ESSA applications and based the rankings on whether states incorporated three criteria that the authors said were important for accountability. The new federal law requires that states identify the lowest-performing 5 percent of schools and measure their performance and improvement, starting in fall 2018. The three criteria are:

Did states assign annual ratings to schools that are clear and intuitive for parents, educators, and the public? In other words, use something like an A to F grade.

Did they encourage schools to focus on all students, not just their low performers? Under the previous No Child Left Behind law, states measured progress by the percentage of students who scored proficient in math and English language arts, compared with students in the same grade the year before. This led states to focus on “bubble kids,” those just below states’ proficiency cutoffs, Fordham wrote, instead of all students. To focus on all students’ improvement, Fordham said states should track the growth of individual students’ scores from year to year and create an index that credits gains below and above proficiency. Fordham gave 23 states an excellent rating.

Did they fairly measure and judge all schools, including those with high rates of poverty? Crediting improvement is critical, especially for high-poverty schools that might score low in achievement, the report said. Accurately assessing the true performance of schools “can’t be done unless it’s possible for high-performing, high-poverty schools to actually earn positive ratings,” the report said. To get a high rating, Fordham required academic growth to make up at least 50 percent of a state’s accountability system.

Seven states got high grades in all three categories. California, North Dakota and Idaho were the only three states to get low scores in all three.

In fairness to California, it probably deserved a grade of “Incomplete.” The state board’s goal is to adopt a true growth model, tied to the annual growth of individual students’ test scores, by fall of 2018, when its school accountability system will go into effect.

David Sapp, deputy policy director and assistant legal counsel for the state board, said the report also contained a big error. California already has moved away from the old standard of rating achievement based on the percentage of students who scored proficient. The dashboard measures performance in relation to the point identified as minimum proficiency on the Smarter Balanced math and English language arts tests. It measures how far above or below that point students, on average, scored.

“The solution is to adopt a growth model,” Petrilli said in an interview. “When it gets its act together, their rating would go up.”

Finally, instead of counting growth in scores for 50 percent of the basis of its accountability system, Fordham said it counts for 20 percent in California — hence its low score on the third criterion. But the study doesn’t state how it came up with the figure of 20 percent, and neither Kirst nor Sapp could figure that out.

It is true that test scores will count for less than 50 percent in the accountability system, but growth and improvement — Fordham’s goal — account for half of a school’s color rating on each academic and nonacademic indicator on the dashboard.

Comments Policy

lee becker2 years ago2 years ago

I just completed a course at Hillsdale Colledge on K-12 Education: Theory and Practice. “The Progressive Influence on Modern Education.” California and its Progressive Influence must revert back to the old tried and proven the way of teaching our children.

Jonathan Raymond2 years ago2 years ago

Who could forget the scene in the Wizard of Oz when Toto pulls back the curtain to reveal the “Wizard.” “Pay no attention to that man,” booms the voice.
First Bellwether and now Fordham espousing the virtues of accountability systems in education. When will we learn that narrow measures of student learning and simple labels about the quality of schools aren’t about continuous improvement – so needed in public education?
This is why Michael … Read More

Who could forget the scene in the Wizard of Oz when Toto pulls back the curtain to reveal the “Wizard.” “Pay no attention to that man,” booms the voice.
First Bellwether and now Fordham espousing the virtues of accountability systems in education. When will we learn that narrow measures of student learning and simple labels about the quality of schools aren’t about continuous improvement – so needed in public education?
This is why Michael Fullan talks about accountability being the wrong driver. Instead, we should be focusing energy and resources on improving instruction and building collaboration among students, teachers, and families. Where the culture, climate and conditions of schools and classrooms are engaging and welcoming to students and parents, where students feel safe, have a sense of belonging, feel trusted and respected and even loved – you see achievement. This is the great secret the billion dollar testing and textbook companies don’t want us to know and which Washington-based institutions like Bellwether and Fordham subscribe. It’s time to pull back the curtain on this grand charade. California should continue to sail in a new direction, avoid quick and simplistic fixes, and look to examples like CORE for new learnings.

Juan Yniguez2 years ago2 years ago

If we approach the issue from a consumer (taxpayer, family) perspective, Fordham is correct. The consumer needs to have a few easy-to-understand measures that quickly summarize a school’s status. I’ll guarantee you, the consumer has neither the time or inclination to wade through a myriad of indicators. At best, they are for practioners and policy makers. At worst, they obfuscate the issue.

Gerald Arcuri2 years ago2 years ago

Given California's upside-down relationship between spending per pupil and academic performance, one might hope that they would be willing to listen and learn from outside analysis. Alas, I have lived in this state for 53 years, and I have rarely seen California civic leaders exhibit such humility and open-mindedness. You could be standing in the road screaming that the bridge is out ahead, and the California establishment elites would be defending their right to drive … Read More

Given California’s upside-down relationship between spending per pupil and academic performance, one might hope that they would be willing to listen and learn from outside analysis. Alas, I have lived in this state for 53 years, and I have rarely seen California civic leaders exhibit such humility and open-mindedness. You could be standing in the road screaming that the bridge is out ahead, and the California establishment elites would be defending their right to drive as they saw fit as they hurtled past you. They know better. They are the progressives.

“We’re not communicating proficiency,” Mike Kirst, president of the California State Board of Education said in an interview this week. “That’s a federal word that they use and we’re not. It’s just not part of the thinking here.”

Paul Richman2 years ago2 years ago

Seems like a lot of space to devote to a report that has errors and a narrow focus. I was especially struck by the Institute president's wholehearted endorsement of "easy-to-understand labels." Labels are part of what has been wrong with past approaches to accountability, and "easy" to the extent it oversimplifies is not helpful, either. We know students and parents care about many factors that contribute to a school's success, and California's system is … Read More

Seems like a lot of space to devote to a report that has errors and a narrow focus. I was especially struck by the Institute president’s wholehearted endorsement of “easy-to-understand labels.” Labels are part of what has been wrong with past approaches to accountability, and “easy” to the extent it oversimplifies is not helpful, either. We know students and parents care about many factors that contribute to a school’s success, and California’s system is trying to have fidelity to these multiple measures by not then boiling it all down to an easy label or number or letter grade that doesn’t really serve a purpose in helping to improve the school. A student’s report card covering multiple subjects doesn’t condensed into a single grade or label, why should a school’s?

John Fensterwald2 years ago2 years ago

Paul: Fordham reports that it gave administrators of the California Department of Education an opportunity to respond to any potential inaccuracies before publication, but the department did not – other than to criticize the focus of the report.

MS2 years ago2 years ago

I would ask why the CDE is expected to review every publication by obviously partisan organizations that have a track record of supporting policies that are not researched based. Bellwether and Fordham are barely different orgs and employ few actual educators. Anyone who has spent time as a teacher, principal, or implementer of policy can tell you that what they espouse does not make sense.

John Fensterwald2 years ago2 years ago

MS:The department can respond to any organization it chooses. However, CDE did answer Fordham, which sent a draft of its ranking of the department. In the response, CDE chose to criticize Fordham for its narrow lens in evaluating the state's school improvement efforts under ESSA but not make any corrections to Fordham's findings. If I were CDE, I might have done both.
Feel free to disagree with Bellwether's and Fordham's approach to education issues, but … Read More

MS:The department can respond to any organization it chooses. However, CDE did answer Fordham, which sent a draft of its ranking of the department. In the response, CDE chose to criticize Fordham for its narrow lens in evaluating the state’s school improvement efforts under ESSA but not make any corrections to Fordham’s findings. If I were CDE, I might have done both.

Feel free to disagree with Bellwether’s and Fordham’s approach to education issues, but I would say it’s simplistic and not accurate to label them partisan. And “conservative” and “liberal” distinctions often defy easy characterization, depending on the issue.

el2 years ago2 years ago

“User Friendly” is good for say making it easy for people to evaluate a school without having to visit it or talk to people about it, but it doesn’t necessarily do a good job of conveying whether your child will be successful there or whether they are doing well with the student population they have. Education is alas complicated, messy, and involves 13 years of changing inputs.