Thursday, November 6, 2014

Most teacher evaluation protocols prescribe a
fairly rigorous process for observers to establish an acceptable level of
inter-rater agreement among their peers when rating the categories of their
District’s teacher practice instrument. Though protocols may vary, a vast
majority involve observers reviewing a series of video scenarios and ranking
each video according to specific look-fors embedded in their teacher evaluation
rubric.

Unfortunately, many school systems and states have
witnessed inflated scores for the teacher practice portion of the teacher
evaluation protocol even though observers had achieved
“certification” with their chosen instrument. Why the inflated scores? Theories
abound ranging from threats of teacher grievances to staged “dog and pony”
performances during announced observations. Regardless of the theory, the net
effect of the millions of dollars invested in teacher evaluation training to
improve instruction has been negligible at best.

How can school systems establish an acceptable
level of fidelity between the key “look-fors” of their teacher evaluation instrument
and actual teaching behaviors? One method is to
begin triangulating the teacher evaluation scores using additional sources.
For example, districts using any version of the Danielson Framework for
Teaching can triangulate their results using informal walkthrough data and
teacher lesson plan submission data to determine the degree of alignment among the
three data sources. Provided below is a sample triangulation that uses the
empirically-validated LoTi (Levels of Teaching Innovation) Framework to
triangulate Danielson Evaluation Practice scores, H.E.A.T. Walkthrough rating
scores, and lesson plan submission scores.

LoTi
Level Ratings

Danielson
Evaluation Practice Ratings

H.E.A.T.
Walkthroughs (LoTi Level)

Lesson
Plan Submissions (LoTi Level)

Danielson
Evaluation Practice Scores

Level
0 - Nonuse

Unsatisfactory

2%

0%

0%

Level
1 - Awareness

Basic

8%

6%

3%

Level
2 - Exploration

Basic

28%

26%

5%

Level
3 – Infusion

Proficient

46%

52%

72%

Level
4 – Integration

Distinguished

8%

12%

12%

Level
5 – Expansion

Distinguished

6%

2%

4%

Level
6 - Refinement

Distinguished

2%

2%

4%

A cursory glance of this table reveals a dramatic gap between walkthrough and lesson plan submission scores and the Danielson Evaluation Practice scores based on the LoTi Framework. This scenario describes the majority of school systems during the
initial year of formal implementation of their teacher evaluation instrument.
Moving into Year 2 implementation and beyond, the goal might be to increase the
alignment among the three indicators by 10% annually and over time begin to
document increased fidelity between the instructional (How we plan
instruction) and operational (How we actually teach) curricula. Only by
following a model of triangulation can the teacher evaluation practice score
possess any real clout in terms of quantifying growth as it pertains to improved
professional practice in the classroom.