Thursday, July 26, 2012

Nils Rauhut and Tziporah Kasachkoff “Everything you always wanted to know about multiple choice question in philosophy—but were afraid to ask.”

Nils: When I came to philosophy, I thought multiple choice
questions were a violation of the spirit of philosophy. But really, multiple
choice questions do things that no other assessment cannot do.

Tziporah: I always thought multiple choice questions were
appropriate for philosophy. After all, we take the GRE. But I did have problems
with essay exams, because the research shows that all sorts of arbitrary things
bias essay grading, including penmanship, the name that appears on the top of
the test, and framing effects. Also, essay exams are impractical.

Audience guy: I’m interested in multiple choice questions as
learning events. Research has shown that mcq with feedback improves performance
on essays down the line.

Tips:

·Good multiple choice questions take time to
write.

·It really helps to have someone else take your
multiple choice tests. If you can’t give your test to a colleague, give it to
yourself 10 days later.

·If a large % of the class gets it wrong, there
is something wrong with the question or the way you teach the material.

·If students identify an ambiguity, be humble
with t. Just because you are the author of a question, doesn’t mean you are the
final judge on what the correct answer is.

·Don’t ever put in wildly implausible answers.

The literature refers to the part of the question that the
answers are about is called the stem.
The false answers are called the distracters..

A bad multiple choice question.

What is an argument in philosophy?

a.A factual disagreement between people

b.Giving reasons for belief

c.A shouting match

d.Any verbal attempt to persuade

Problem, according to Tziporah: (c) is too implausible to be
a good distracter.

First question: What do I want this multiple choice test to
do?

Tests to check if
students have done the reading.

Nils uses basic reading comp questions (7-8 minutes) at the
start of classes, and have them grade their peers tests, to embarrass the
students who haven’t done the readings. You need at least 10 questions for this
to work.

Questions to ask yourself: Is it really the case that
someone hasn’t done the reading will do badly and someone who has done the
reading will do well.

He hands out a sample quiz. Could a student who did the
reading actually ace this quiz? Paul: This quiz should be open book.

Tests to check higher
level of comprehension.

Tziporah: gives an example of a forking sequence of
questions, where you have to justify your answer to the first question in the
second question. Students don’t get questions for the right answer unless they
also answer the reason question correctly. Students also get no credit for
wrong answers for the first question, no matter how good the answer to the
second question is.