designing test questions

They’re the kind of questions that promote thinking and result in sophisticated intellectual development. They’re the kind of questions teachers aspire to ask students, but, according to research, these types of questions aren’t the typical ones found on most course exams. Part of the disconnect between these aspirations and the actualities results from the difficulty of writing questions that test higher-order thinking skills.

The goal of any well-constructed test is to test students’ expertise on a topic and not their test-taking skills. We need to eliminate as many flaws in our questions as we can to “provide a level playing field for testwise and not-so-testwise students. The probability of answering a question correctly should relate to an examinee’s expertise on the topic and should not relate to their expertise on test-taking strategies.” (NMBE, 2001, p 19)

I remember with horror and embarrassment the first multiple-choice exam I wrote. I didn’t think the students were taking my course all that seriously, so I decided to use the first exam to show just how substantive the content really was. I wrote long, complicated stems and followed them with multiple answer options and various combinations of them. And it worked. Students did poorly on the exam. I was pleased until I returned the test on what turned out to be one of the longest class periods of my teaching career. I desperately needed the advice that follows here.

As Ron Berk (known for his pithy humor) observes, the multiple-choice question “holds world records in the categories of most popular, most unpopular, most used, most misused, most loved and most hated.” According to one source I read, multiple-choice questions were first used around the time of World War I to measure the abilities of new Army recruits. As class sizes have grown and the demands on teacher time expanded, they have become the favorite testing tool in higher education.