Documents show that system ignored flawed information in cheating investigation

Erica L. Green, The Baltimore Sun

When Baltimore City Councilwoman Mary Pat Clarke visited Abbottston Elementary after news broke that the school had cheated on state testing for students in 2009, she despaired because she never believed Principal Angela Faltz, whom she had known for decades, could cheat her children or her community.

"But there was nothing to be done about it at the time," Clarke recalled in an interview Friday. "I was able to give Dr. Faltz a hug, and then we never saw her again."

Now, after lengthy investigations, Faltz and Assistant Principal Marcy Isaac have been recommended for reinstatement, with full back pay, by independent hearing officers. The findings not only vindicated the administrators but called into question the school system's findings of cheating at three schools, as investigations into testing improprieties continue at 16 additional schools.

According to documents obtained by The Baltimore Sun this week, independent hearing officers — attorneys hired by the system to render facts and findings in personnel hearings — found the school system's investigation, triggered by plunging student test scores, to be fundamentally flawed.

"While there is a healthy dose of speculation, suggestion and suspicion that cheating occurred in 2009, there is a lack of credible evidence that cheating actually occurred," one hearing officer wrote. "Not one witness was able to testify with certainty that they knew what happened in Abbottston. To the contrary, witness after witness stated that they did not know what happened at Abbottston."

The hearing officers also said city schools CEO Andrés Alonso knew there were problems with a state analysis of eraser markings in test books but moved ahead anyway to dismiss the principals last year.

The findings also raise questions about how the system determines whether a school cheated and who is responsible.

This was not the first time a city principal has been removed for irregular test results despite a lack of evidence about who was responsible for the cheating, according to a former investigator for the school system.

Jose Rosado, who left the system last June after three years, investigated the case of George Washington Elementary in 2010, the year that Alonso announced he had revoked the professional license of its principal, Susan Burgess.

The school, which had earned a Blue Ribbon award for its achievement, was confirmed to have cheated by a state investigation. In an interview, Burgess denied that she was responsible for the thousands of erasure marks the state found on tests.

Alonso said at the time that if she didn't know cheating had occurred, she should have — a standard he would continue to uphold for principals. Burgess did not fight the system's decision.

Rosado said Burgess acknowledged being in the testing rooms, which is not a breach of protocol. After several interviews with staff members, Rosado said, he informed the system that there was no evidence to suggest that Burgess knew of or did anything wrong.

"There was no evidence that pointed to her," Rosado recalled in a recent interview. "Just seeing that woman in there, and the erasure analysis, they got rid of her. The erasing could have been done in a classroom. As little evidence as they had on her, they made an example of that poor woman."

Hundreds of documents associated with the case of Faltz and Isaac illustrated how key players in the city and state's initial investigation stopped short of reaching any conclusions about whether cheating had occurred when scores dropped drastically in 2010. The officials also reserved judgment about who or what played a part in the declines.

Still, Alonso was not convinced that cheating hadn't occurred. He maintained that position even though interviews conducted with 26 school employees by his internal investigator found that no staff member had witnessed test tampering.

School system officials argued that the school's benchmark scores — which are used to predict how students will perform on the Maryland School Assessment — were so different from the MSA results that the principals should have known something wasn't right. Moreover, other schools that had similar obstacles, such as high student turnover, didn't experience the same test score drops.

Alonso declined to comment Friday. The school system does not comment on personnel hearings.

But according to his testimony before the hearing officers, Alonso "agrees that benchmark testing ... does not in itself, prove there was cheating," according to the report. The report adds that he gave interviews with the 26 school employees "their proper weight, particularly given that the same kinds of statements were made about another school where it was ultimately determined that cheating had occurred."

The most troubling aspect of the school system investigation, according to the hearing officers, was that the system built its case on a faulty erasure analysis. In his announcement that Abbottston had cheated last year, Alonso said the analysis showed patterns of erasures from wrong to right answers "beyond the realm of probability."

In their findings, the hearing officers called the erasure analysis "incompetent" and "crude," and outside experts have called it an embarrassment to the scientific profession. The erasure analysis, conducted by a longtime state Education Department official, was done manually in a living room. Any documentation that could have authenticated it was destroyed.

Although the city said in its news release last year that all 485 of Abbottston's test booklets were reviewed in the analysis, state Education Department reports indicated that only 167 were reviewed, according to the hearing report.

School system officials declined to comment on the discrepancy.

Most troubling, the investigators said, was the appearance that the state official had predetermined her results. "I want to be sure that [the Maryland State Department of Education] is confident in the findings and that [the city school system] has empirical data to support my review," the official wrote in an email one year before submitting her results. "The BIG question is — How much evidence is enough?"

City school investigators said that without the erasure analysis, the school's test scores would not have been invalidated, according to the report.

Robert Wilson, the lead investigator of a cheating scandal in Atlanta, considered the largest in the history of U.S. public education, said the Abbottston scores should never have been invalidated based on the analysis because it was so flawed. He reviewed hearing documents in the Baltimore case.

Wilson led the 2011 investigation that uncovered rampant cheating in Atlanta public schools — where his team issued a 171-page report identifying 150 educators, 82 of whom confessed to cheating.

Atlanta's erasure analysis consisted of reviewing more than 100,000 test booklets to establish a benchmark, electronically scanning the test booklets and a manual analysis to verify the results.

"If you showed me 167 booklets, and I did exactly what she did, I might be able to say, 'This is a hell of a lot of erasures,' but I'd have to have a lot more than that," Wilson said.

"I am surprised that someone who claims to a psychometrician would have been willing to draw conclusions from such unreliable work," he added. "That the Baltimore City Public Schools would have relied upon it is even more surprising."

Lawyers for Faltz and Isaac will next give oral arguments this month to the city school board, which will decide whether to uphold or reject the recommendation. Clarke, whose district includes Abbottston, said she hopes that the board will be as deliberate as the hearing officers in considering what's been presented when it decides whether to accept or reject their recommendations.

"It broke our hearts what happened, and we rejoice that we've all been vindicated," Clarke said. "While I'm a strong supporter of Dr. Alonso, I'm also a strong supporter of Dr. Faltz. After all, it was the school system that said take a closer look, right? Well, this is the outcome."