Chapter VIChoosing or designing an instrument to guide peer observation

The purpose of a peer observation will determine to a great extent the type of instrument that will be appropriate in conducting it. Instruments range from the highly restrictive to the open-ended, and any of them might be effective, depending on the given circumstances.

Although they can be useful for distilling a lot of information in a short form, checklists tend to be the most prescriptive and restrictive method of communicating observation data. After all, many of us have taken courses from highly effective teachers who had few (perhaps none) of the textbook behaviors associated with good teaching. And many of us have

taken courses from highly ineffective teachers who serve as textbook examples of “good teaching behaviors.” (Return to top)

So, should we simply reject the checklist?

No. In fact, the checklist can be a very effective when used with other instruments, such as a written analysis (see below). While it may not tell the whole story, a checklist can be a good way to identify potential reasons for why a given instructor is effective or not. It is particularly good for a first-time observation or for

troubleshooting a problematic situation. The data collected can point the observer to places to look for a more in-depth analysis. A wide representation of effective behaviors might help explain why a given instructor is successful; a dearth of those behaviors might help explain why an instructor is struggling.

The assumption behind this type of instrument is that sound teaching behaviors lead to effective teaching. Using visual support, asking good questions, waiting for answers, calling on students by name: all of these have been shown through educational research to enhance student learning. An instructor who does these things, the logic goes, is doing a good job.

Here are some sample items in an instrument of this sort.

Does the instructor use active learning techniques?

Do students have chances to apply concepts during class?

The observer has the opportunity to check whether these behaviors were observed and then has room to make comments regarding this behavior. A sample instrument of this type can be found here. (Return to top)

How about ratings using a Likert scale?

These are by far the most common instruments used for peer observation, just as they are used for student evaluation of teaching. We do not recommend the use of a Likert scale instrument for formative purposes. These instruments can have a judgmental tone that may

not be effective for giving feedback about teaching. If a Likert-style instrument is used, it needs to be accompanied by written analysis to ensure that the feedback received is informative rather than merely judgmental. (Return to top)

Is the written analysis a superior instrument?

The written analysis requires more work on the part of the observer, but whether it is used as a stand-alone document or in accompaniment to other instruments, it is an essential piece of any peer observation. The analytical process conducted either through a checklist or a ratings instrument needs to be incorporated into a more holistic view, since sometimes the whole can be

greater or less than the sum of its parts. The written analysis also allows the observer to address the instructor’s goals for that particular session, which is a level of tailoring that the other instruments do not allow.

However, the written analysis is not merely an open-ended document, expressing an observer’s opinion. It should be guided by the agreed-upon criteria (either the individual’s or the department’s), so as to structure a peer’s observation and report.(Return to top)

What is an ethnography-
style peer observation?

In some cases the function of the observation is to collect observed data from the classroom as a way of beginning a conversation between instructor and observer. An observer engaging in this process tries simply to capture as many details of the event as possible, as objectively as possible. This means transcribing what happens during the class meeting.

The resulting document, which looks a bit like an ethnography, can be a powerful way of providing formative feedback to an instructor, and is much less threatening than a videotape. The observer brings the document to the post-observation meeting and goes through it with the instructor. This allows for a conversation about specific choices the instructor made, and alternatives, if appropriate. A sample instrument of this type can be found here. (Return to top)

Are any of these more appropriate for one type of evaluation (formative) over the other (summative)?

Any of these instruments, with variations, can serve either type of process. Perhaps the most important thing to keep in mind is that for summative evaluation, in which careers are at stake, a combination of approaches is more likely to generate a valid and reliable evaluation. (Return to top)