Chicago Teachers See Value in New Evaluations, But Eschew Test Scores

New teachers in Chicago face higher expectations under a teacher-evaluation system rolled out last school year. And while teachers appear to find feedback generated by the system helpful, they remain deeply wary of its emphasis on student achievement.

That's the takeaway from two new data dumps on teacher evaluations in Chicago, both released this morning in the Windy City.

About 40 percent of nontenured teachers scored at the "developing" phase, the second-lowest of four categories on the system, while 48 percent were deemed proficient, according to results from the Chicago school district. About 3 percent of nontenured teachers were deemed ineffective.

The results are notable partly because of the context surrounding the new system, called REACH, which represents the first overhaul of teacher evaluations in Chicago in 40 years. Teacher evaluation was among the precipitating factors in last year's seven-day teachers' strike. And the recent closing of 49 schools generated several rallies throughout the city and has contributed to teachers' general distrust of the district administration.

Nevertheless, the Chicago Teachers Union and the district mapped out many details of the new review system on a labor-management committee. Barbara Byrd-Bennett, the district's CEO, said that devising a system that helped teachers get better was first and foremost in that committee's mind. She drew a contrast between former "checklist" teacher evaluation systems, and REACH, which requires principals to provide evidence behind the scores they assign and expects them to help teachers improve.

The old checklists, she said, focused on "You didn't do well, you didn't do this, you didn't, you didn't, you didn't," Byrd-Bennett told reporters at a briefing. "This system provides teacher with pinpoint areas of improvement and strength."

Karen Lewis, the president of the CTU, was much less positive about the first-year results. She focused on a separate analysis released today by the Consortium on Chicago School Research, in which more than half of teachers surveyed said tests were weighted too heavily in the reviews.

"This is what we've been saying all along—the evaluation system is deeply flawed. Teachers don't like testing being part of their evaluations, not because we think that student outcomes are unimportant, but because these tests do not indicate how teachers are contributing to learning," Lewis said in a statement. "The test is just a snapshot; classroom observation is still the best way to measure teacher performance."

Illinois, like many other states, recently approved legislation to revamp its teacher evaluations. Chicago's system was put on an expedited timeline to comply with the 2010 law, and REACH rolled out just weeks after the strike concluded. In its first year, 2012-13, the system covered approximately 4,200 nontenured teachers.

REACH relies heavily on observations of teachers based on the popular Framework for Teaching. Most, but not all, teachers also have a measure of student achievement factored in.

Here's the money graph from the district's presentation to reporters. Compare the results to four years earlier.

The results generally show better differentiation in teacher performance, CPS officials said, noting that it makes sense that more nontenured teachers new to the profession would still be "developing" rather than "excellent."

Even before REACH took effect, the graphic shows, teachers' reviews had been getting a bit more difficult, a fact CPS officials attributed partly to a pilot evaluation program that affected about 100 schools.

Tenured teachers will begin receiving formal evaluations under REACH starting this school year. But 91 percent of tenured teachers also received at least one observation under REACH, and those results showed some general areas of strength and weakness.

For example, teacher performance on the instructional domain, which accounts for 40 percent of the observation score, was generally weaker than on professionalism or planning. About a third of nontenured teachers and a quarter of tenured ones had "basic" or "unsatisfactory" scores on their ability to use questioning and discussion techniques. A similar proportion struggled to use assessments in instruction. Chicago officials said the findings would be used to improve professional development.

Scores on the student-achievement component of REACH weren't broken out by the district.

Teachers Supportive?

In a separately released report, more than three quarters of the district's teachers said that the evaluation system has the potential to make them better at their jobs. But they remained deeply concerned about the part of the evaluation based on student test scores.

The report is from the Consortium on Chicago School Research, a group based at the University of Chicago. (CCSR also studied Chicago's pilot evaluation program a few years back.) It's based on surveys of some 700 principals and about 19,000 teachers. (Remember, most tenured teachers received at least one observation on the new system even though it only officially rolls out for them in 2013-14.)

Teachers and principals generally favored the observations, which are 45 minutes long and can occur up to four times annually for a nontenured teacher. Seventy-six percent of teachers said that the process encouraged their professional growth, and as the figure below indicates, they generally felt that their principals did a fair job in carrying out the system and providing feedback.

"It is apparent that teachers and principals engaged in this work in good faith. They were sincere about it, they believed in it, they took it seriously," said Paulette Poncelet, the district's executive director for talent development.

Some teachers, though, reported feeling uncomfortable asking their principals for help on specific practices, fearing it would come back to haunt them during the formal evaluation process. (The tension between the "formative" and "summative" purposes of teacher-evaluation systems has been a subject of debate for some time.)

CTU officials said they still felt principals were not trained enough to help coach teachers effectively.

In general, teachers did not like the student-growth part of the evaluation system. In Chicago, this can be based on several different measures: growth on student tests; a gain score on a district-developed "performance task," which are written in part by teachers; on "expected gains" on a suit of ACT-related tests; or on a measure of average schoolwide literacy growth.

Fully 57 percent of teachers said they thought that REACH relied too heavily on tests. The finding is a bit tricky to interpret because the CCSR analysis also found that many teachers were confused about how the evaluation pieces added up into a final score. (Some teachers mistakenly thought their whole evaluation would be based on student scores. In fact, they currently top out at 25 percent.)

Administrators, too, said they often weren't sure about how all the pieces were assembled to arrive at each teacher's final score.

Two-thirds of administrators also said that the system took up too much time. The CCSR report indicates that principals reportedly spent an average of six hours on each teacher's evaluation and related debriefs. Byrd-Bennett highlighted that as one area to work on. On the one hand, principals felt that multiple evaluations were helpful. On the other, they feel burdened by administrative duties. Chicago must prioritize ways of elevating principals' role as instructional leaders, she said.

"If our principals have lost their way, we need to help them do the work for which they were hired," she said.

The Joyce Foundation provided support for the CCSR report. (The philanthropy also helps support coverage of teacher quality in Education Week.)

Categories:

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.

Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.