Study: Charter Evaluations Have Room for Improvement

Evaluating charter school performance continues to be an inexact
science, but one that can be mastered when overseers have more
resources to conduct thorough reviews, a national study released here
last week concludes.

Even charter school authorizers with ragged evaluation systems
manage to stumble into taking the "correct" action when it comes to
weighing a charter school's fate, the report says. And authorizers
aren't hesitant to close schools that require that drastic step.

"The decisions ultimately come out correctly, but not on the basis
of the kind of process we would like to see," said Bryan C. Hassel, a
co-author of "High-Stakes: Findings From a National Study of
Life-or-Death Decisions by Charter School Authorizers."

Mr. Hassel, the president of Public Impact, a Chapel Hill, N.C.-based
education policy consulting firm, said he wanted to examine whether
charter authorizers were shutting down schools that were not performing
and how those decisions were being made. The Smith Richardson
Foundation, of Westport, Conn., paid for the study, which is co-written
by Meagan Batdorff, a consultant with Public Impact.

The study compiled 506 "high-stakes" decisions—renewing, not
renewing, or revoking charter school contracts—made by charter
authorizers nationwide in 2001. Using that list, 50 randomly selected
cases were reviewed through interviews and analysis of documents and
media accounts.

Among the study's main findings was that roughly one out of six
high-stakes decisions led to school closures. About 84 percent of the
decisions were renewals, while 16 percent resulted in schools' loss of
their contracts.

Mr. Hassel, who presented his study at the Brookings Institution in
Washington Feb. 18, believes the figures represent a "fairly high rate"
of closure because most of the high-stakes decisions made were sound.
In the 50 case studies, the researchers found that only one school was
not closed despite signs of "underperformance," while supporting
evidence was not clear in the closures of four other schools.

The report adds that many charter authorizers based their decisions
on inadequate information or had failed to develop clear expectations
with the schools. In more than half the cases, authorizers did not make
"merit based" comparisons of evidence and agreed-upon goals. Political
pressure was more commonly apparent, it says, in charter school
decisions made by local school boards.

The researchers recommend that states provide more money to
strengthen charter authorizers' evaluation capabilities. Attracting
authorizers other than local school boards would be beneficial, they
argue, because universities and state education agencies—which
issue charters in some states—tend to develop better evaluation
programs.

"If you've got a lot of schools to oversee, you can't just do it on
the back of an envelope," Mr. Hassel said. "You have to develop a
system and be more deliberate about it."

New Bureaucracy?

Robin Lake, the associate director of the Center on Reinventing
Public Education at the University of Washington in Seattle, says that
authorizers should not expect more state funding until the evaluation
process is improved.

Ms. Lake suggested during the Brookings Institution discussion that
states set minimum standards for oversight. Authorizers should be
randomly audited to ensure that they are conducting adequate reviews,
she added.

Mark Cannon, the executive director of the National Association of
Charter School Authorizers, balked at the notion of drafting state
policies that provide evaluation guidelines for authorizers. The
Alexandria, Va.-based group represents 50 authorizing bodies.

While the association encourages charter authorizers to use a
rigorous review process, he said, more state compliance rules "could
stifle the innovation that charter schools tend to promote."

No one wants to create another publicly financed bureaucracy, said
Tom Mooney, the president of the Ohio Federation of Teachers, an
affiliate of the American Federation of Teachers. But Mr. Mooney, a
chief critic of his state's charter school system, said in a telephone
interview that adopting state standards for charter authorizers could
provide a better accountability process.

"Most people want safeguards of quality and safeguards of how money
is being spent," he said.

Vol. 23, Issue 24, Page 9

Published in Print: February 25, 2004, as Study: Charter Evaluations Have Room for Improvement

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.