For better multiple-choice tests, avoid tricky questions, study finds

a) assessing a student's mastery of facts and concepts;
b) helping students learn and retain facts and concepts.

While some educators might see this as a trick question, the correct answer appears to be:

c) all of the above, suggests new research from Washington University in St. Louis.

"Although people often think about multiple-choice tests as tools for assessment, they can also be used to facilitate learning," said Andrew Butler, a cognitive psychologist in Arts & Sciences who studies the brain processes behind learning and recall. "The act of retrieving information strengthens memory for that information, leading to better long-term retention, and changes the representation of the information, creating deeper understanding."

Butler's study, published in the September issue of Journal of Applied Research in Memory and Cognition, offers straightforward tips for constructing multiple-choice questions that are effective at both assessing current knowledge and strengthening ongoing learning.

Among key findings, educators should never include trick questions or offer "all of the above" or "none of the above" options among the list of possible answers.

Research on the format of multiple-choice questions is important, Butler noted, because the tests are widely used throughout the world, especially in the United States where they originated as part of early efforts to measure intelligence.

Fueled in the beginning by the need for an efficient way to measure characteristics of World War I soldiers and booming student enrollments, multiple-choice tests now influence important life decisions in areas such as college placement, workplace hiring, career advancement and even online dating.

As an associate professor in the Departments of Education and of Psychological & Brain Sciences, both in Arts & Sciences, Butler conducts research that explores the malleability of memory—the cognitive processes and mechanisms that cause memories to change or remain stable over time.

Taking any form of test has the potential to alter our understanding of a topic, he said, because the process of recalling information requires important details to be freshly reconstructed from related memories.

While multiple-choice testing, especially repeated testing, has the potential to strengthen our recall, a poorly formatted test question can have the opposite effect, Butler said. Such an ill-formed question can muddy our recollection of the correct answer and reinforcing memories for inaccurate "distractor" answers, he added.

Butler's research review confirms that proper question formatting and presentation are critical to creating effective multiple-choice tests. It also suggests that many widely used multiple-choice tests still include lots of questions that fail to comply with research-based best practices.

"Fortunately, the best practices for creating multiple-choice tests that effectively assess understanding are much the same as those for supporting student learning," Butler said.

Create questions with simple formats. Complex question-and-answer formats have become popular as educators look for ways to test for deeper understanding and higher-level learning. Examples include offering answers such as "A and B, but not C," or allowing repeated answers until correct choice is made. Such formats may be detrimental to assessment since guessing is encouraged and processing may focus more on parsing question context and less on recalling correct information. Some complex formats, such as confidence-rated answers, may offer benefits, but more research is needed.

Create questions that engage "real world" cognitive processes. To truly test abilities, questions must be structured so correct answers require use of the specific cognitive processes necessary to address similar problems in the real world. Questions that require higher-order thinking will enhance learning and improve future performance. For example, a multiple-choice question could require a test-taker to contrast two concepts (Which of the following is a way in which hawks differ from eagles?) or analyze a set of conditions to make a decision (Given a patient symptoms, which of the following diagnoses is most likely?).

Avoid using "none of the above" and "all of the above" as answer choices. When "none-of-the-above" is correct, students may not need to retrieve correct information to answer the question and they are exposed a lot of incorrect information. Using "all of the above" exposes students to a lot of correct information, but answers may be more obvious, robbing students of potential learning that comes from recall processing. Both question types can be detrimental to accurate assessment and potential benefits to learning are small.

Use three plausible response options. Question difficulty increases with each answer option offered. Students who correctly answer more difficult questions may learn more from rising to the challenge, but questions that offer too many plausible answers can have a negative effect on both learning and assessment. Use the Goldilocks principle: not too many, or too few.

Make the test challenging, but not too difficult. Create tests that are hard enough to reveal how well students know the material, but easy enough that a majority (80 percent) get a passing grade. Retrieving information and answering questions correctly reinforces student learning; failing to answer correctly may strengthen memories for misinformation. Challenge students, but allow them to succeed.

Finally, because multiple choice questions expose students to lots of plausibly presented false information, it's important for students to review answers after grading is completed. Feedback enables test-takers to correct errors and avoid internalizing incorrect information. It also strengthens learning around correct answers that were low-confidence guesses at test time.

"One takeaway from these recommendations is that the most effective multiple-choice items get students to think in ways that are productive for learning and enable valid measurement of whether they have acquired the desired skills and knowledge," Butler said. "To maximize both effectiveness and efficiency, it is also best to keep the process of answering multiple-choice items simple—added complexity often has a negative effect on both learning and assessment."

More information:
Andrew C. Butler. Multiple-Choice Testing in Education: Are the Best Practices for Assessment Also Good for Learning?, Journal of Applied Research in Memory and Cognition (2018). DOI: 10.1016/j.jarmac.2018.07.002

Related Stories

Can standardized tests, such as those created in response to the Common Core, enhance education instead of just assessing it? For standardized testing to benefit students, researchers recommend that the tests are used as ...

(Phys.org) —A team of scientists who specialize in multiple-choice test assessment at Brock and Trent University's in Canada has conducted a study to find out if students could use Benford's law to help them pass multiple-choice ...

Healthy adults who learn information more quickly than their peers also have better long-term retention for the material despite spending less time studying it, a new study from psychologists at Washington University in St. ...

Your squad, or social group, can have a huge impact on how you view the world. Now, new research shows that the people you hang out or work with might also impact how well you can identify fact from fiction.

For many students, essay tests are a source of dread and anxiety. But for professors, these tests provide an excellent way to assess a student's depth of knowledge and critical-thinking skills. At the American Physiological ...

A new study by the University of British Columbia shows that teachers don't have to test everything they want their students to remember—as long as the knowledge they want to convey fits together well, and the test questions ...

Recommended for you

New photonic tools for medical imaging can be used to understand the nonlinear behavior of laser light in human blood for theranostic applications. When light enters biological fluids it is quickly scattered, however, some ...

One of the ocean's little known carnivores has been allocated a new place in the evolutionary tree of life after scientists discovered its unmistakable resemblance with other sea-floor dwelling creatures.

In research that casts cells as curators of their own history, Dana-Farber Cancer Institute scientists have discovered that adult tissues retain a memory, inscribed on their DNA, of the embryonic cells from which they arose. ...

Using radial velocity method and direct imaging, astronomers have found a compact companion of a nearby star known as HD 118475. The finding is detailed in a paper published March 11 on the arXiv pre-print server, in which ...

3 comments

Multiple choice tests are not a good way to test anything. They give the false impression of mastery, but students can get by by eliminating 'implausible' answers and picking the one answer left (or simply eliminating as many as possible and then just playing the odds and the psychology of the test maker who will rarely do things like make a test that will have the answer a) 10 times in a row.

*Mastery* of anything is only demonstrated by training someone and then giving them a new task which they have to figure out on their own - with NO aid given. Try that with someone who has just passed a multiple choice test. They will fail to a scary degree.

because the tests are widely used throughout the world, especially in the United States...

And the quality of the United States education system should give a hint as to how sensible multiple choice tests are

Multiple choice (in whatever of the in the article mentioned forms) tests recall - not understanding.

Make the test challenging, but not too difficult. Create tests that are hard enough to reveal how well students know the material, but easy enough that a majority (80 percent) get a passing grade.

No. Do not give a free pass for people who don't understand (or to teachers who are inept at teaching) just because you want to pass 80% of the class. That's a sure fire way of sending people on to college or into the job market that are woefully unprepared.

What does the author think schools do? Give a few years of feel-good environment until the shock of real life destroys these people? That is doing them a massive disservice.