Evaluation of auto-generated distractors in multiple choice questions from a semantic network
Lishan Zhang, Kurt A. VanLehn · Interactive Learning Environments · 2019
Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called “distractors”). We designed an algorithm to generate distractors from a semantic network for four types of multiple choice questions in biology. By recruiting 200 participants from Amazon Mechanical Turk, the machine-generated distractors were compared to human-generated distractors in terms of question difficulty, question discrimination and distractor usefulness. The machine-generated and human-generated distractors performed very closely on all the three measures, suggesting that generating distractors from a semantic network for simple multiple choice questions is a viable method.