SUNetID Login

Interesting Uses of Interactive Questionnaires

Message Number:

22

(5/4/98)

Folks:

Here is an interesting response from Michael Fried, at UC Irvine regarding Message #21 on Teaching Large Classes: Strategies for Improving Student Learning.

He referres to the use of Interactive Questionnaires (IQ's) which he described in a previous message (#17 Improving Student Learning While Saving Faculty Time), more information about which can be accessed through:

His comments about the impact of such questionnaires on student learning is impressive, his comments on their effects on student evaluations of his course (down considerably) is troubling.

Rick Reis
reis@stanford.edu

---------------- 765 words ---------------

Uses of Interactive Questionnaires

I didn't have formal research that supported the statements by Rick's message last week on the research of Graham Gibbs, Centre for Higher Education Practice, Open University, UK. Still, it agrees with the most interesting discoveries (negative and positive) that came from the technology I developed under a Sloan foundation grant called IQs (Interactive Questionnaires). There is a comment at the end of this message explaining IQs. Here are two examples of the discoveries.

1. BRINGING PEER EVALUATION INTO PROBLEM SOLVING:

IQs allow effortless display of the level of approach of all students to the problems students investigate for a grade.Thus, students get peer pressure to respond to the work of other students knowledgeably. Also, at first they get to see that some students are able to follow precisely while the majority are faking it.

EFFECT: Many stop faking their answers, because they will no longer be able to hide behind ineffective or hidden evaluation.

2. BREAKING PROBLEM DIFFICULTIES INTO SMALL PIECES:

Since IQs are dramatically written in a modular form, helping students to parse the pieces of the problem puzzle, student difficulties are much clearer. Further, instead of looking like they are impossible to solve ---because there is no exact problem in the book like one put forth on an IQ---the cognitive problem becomes clear, which is that students have no coherent strategy for many step thinking. Further, they have not practiced many step thinking, so they don't know the "expert's" secret for relaxing at each stage by thinking of just ONE STEP AT A TIME. That is, students give themselves a cognitive overload by holding dear to the memory of all steps of the problem as they attempt the next step. The load gets heavier and heavier.

RESULT 1. MORE ADVANCED PROBLEM SOLVING; LESS CHEATING:

IQs allow for dramatically more advanced problem testing, and there is a dramatic drop in various types of (purposeful and subconscious) cheating. The drop is voluntary---they feel they can't just copy someone else's material. Even if they do copy it, they feel they must still be able to explain it. By the way, this has exactly the effect predicted by Rick's message on the research of Graham Gibbs.

EFFECT 2. POORER TEACHER EVALUATIONS:

Though students demonstrably improve under this procedure--it's not even close--THE EFFECT ON TEACHER EVALUATION GOES THE OTHER WAY.
The most dramatic documented example is from the start of my using serious technology, about eight years ago. I had the HIGHEST RATED course in the physical sciences at UCI before the IQ and Sloan technology. My picture wa s in the local newspapers, and the UCI campus catalog had a one page spread on me. Within a year, the same course was one of the LOWEST RATED after I adopted the e-mail technology. This effect is so dramatic, it rivals the increase in student performance . Eventually, I found ways to mitigate it though my ratings have never climbed back to where they were. Surprisingly, many students like the technology, some calling it "cool" because of how it allows them to type in their answers with everything else au tomatic about logistics for getting their work in.

A LITTLE MORE ABOUT IQs:

An IQ is an evaluation program (test, quiz, attitude) disguised as an e-mail message. Briefly: It is an enhanced interactive exam, offering various aids and guidance to a student who needs it.

(See http://www.math.uci.edu/~mfried/ or URLs above for example Iqs and student interactions with them.)

I send IQs to get interaction with the class about where they are on the material without going into MIDTERM mode. Teaching the modularity of mathematics (or any many-step thinking) is exceptionally hard, so, it is especially fitting that IQs are highl y modular. Thus, they help students focus on analyzing one step at a time. A student takes an IQ at a computer terminal---any terminal with access to their mail account. When he or she finishes, the IQ automatically returns to me by mail for (automatic ) placement in the student's portfolio.

Data from an IQ is in pieces. You can view these with a program, and extract a student (or many students') response(s) to any IQ piece. Therefore, IQs help an instructor focus on small conceptual elements that go wrong in a class. These reports, sent around to students, are especially handy for getting peer responses to student work. I usually make it part of the grading to get each student to evaluate pieces of the work of other students.

This simplifies grading, and raises student standards in agreement with the effect I reported above.