Common Mistakes in Writing MCQs
Problems with Multiple Choice Questions
When multiple choice questions have issues, they usually have to do with Cognitive Load and/or Validity issues. Cognitive Load refers to the amount of thinking it takes to process a question. A question stem is effective when it can deliver the problem to the test taker simply. If a stem can be simple, logical, limited in the use of negatives, and meaningful, it can help reduce the cognitive load on a student. Cognitive load theory recommends avoiding elements of instruction or assessment that will overload students' capacity to consciously process the immediate task on which they are working (Sweller 1998).
A Validity issue occurs when the results of the question don't actually assess the desired outcome. This can be simply because the question is poorly worded, or more often because of Rival Explanations. A Rival Explanation is when there is more than one possible reason why a student selected the wrong answer. As Jason mentioned in his video, a good question will tell you not only if they got it right or wrong, but also why they got it wrong, and also identify what common mistake or misunderstanding they've made.
The example questions below illustrate some common mistakes in writing multiple choice questions that increase cognitive load, and cause validity issues by introducing rival explanations.
What makes up a Multiple Choice Question?
Multiple choice questions have specific terminology used to describe the different parts of a question.
Stem: A question or statement followed by a number of choices or alternatives that answer or complete the question or statement
Choices: All the possible choices or responses to the stem
Distractors (foils): Incorrect choices
Correct answer (key): The correct choice!
Effective Stems
Keep the Stem Simple
Reduce the clutter in the stem and reduce non-essential information. Too much information in a stem can confuse students.
Avoid Negative Phrasing
Unless the learning outcomes are associated with negative phrasing in questions, avoid negatives when possible. Students often have difficulties understanding and processing negative questions.
Make the Stem a Question or Partial Sentence
Reduce the cognitive load of a student by asking a direct question. In the better example, the question is direct and the student can focus on answering the question. In the first example, the student has to hold the partial sentence in memory and complete the questions with each possible answers. If you can't find a way to write a direct question, try to move the blank to the end of the sentence.
Effective Distractors
Use Plausible Distractors
Choose distractors with a purpose. Implausible distractors don't help you identify whether or not your student understands the topic being assessed. In the extreme example below, students can easily remove questions A and D, leaving them with only two possible choices. The number of distractors can vary as long as they are plausible. Don't feel as if every question requires 4-5 choices.
Keep Choice Lengths Consistent
Try to keep a consistent length with the choices you have as distractors. Students can easily guess the correct answer when one choice is particularly wordy and specific while other distractors are brief.
Placement of the Correct Answer
Keep from consistently placing the correct answer as one specific choice. If students find a pattern in which answers are the correct ones (e.g., "c" is usually the right answer or "d" is never the right answer) then they can increase their chances of correctly guessing, providing a rival explanation. When distractors consist of a series of values on a continuum, place the choices in order logically (e.g. numerically, or chronologically).
Keep Grammar Consistent
Good test takers can notice clues to the correct answer when grammar isn't consistent between the stem and the distractors. In the example below, proteins is listed in its plural form, while the stem is asking for a singular substance.
Avoid "All the Above" or "None of the Above" Options
When “all of the above” is used as a choice, test-takers who can identify more than one alternative as correct can select the correct answer even if unsure about other alternative(s). When “none of the above” is used as a choice, test-takers who can eliminate a single option can thereby eliminate a second option. In either case, students can use partial knowledge to arrive at a correct answer, thus not truly assessing whether they knew the whole answer.
The main point
Some of you, while reading the above examples probably thought to yourself, "Yeah, but the students should be smart enough to figure that out." You're right, they should - but that's irrelevant at this point. If your stem or distractor is confusing and the students choose the wrong option, we don't know if they didn't know the material or just got confused, this introducing a validity issue. Remember, the value of the assessment comes from it's ability to accurately tell us whether the students have reached the desired outcomes.
Click Next to learn from some other experts in this field.