Multiple-choice vs free-text code-explaining examination questions
Simon, Susan Snowdon · 2014
The BRACElet project has developed a number of code-explaining questions and thoroughly researched the novice programmer's difficulty in answering them correctly. In a prior study we explored whether students might perform better on multiple-choice code-explaining questions than on free-text code-explaining questions, and concluded that the multiple-choice form led to a perceptible but generally insignificant improvement in students' performance. However, that study compared different cohorts of students, leaving some doubt over its validity. In this study we seek to find a more definitive answer by including multiple-choice and free-text code-explaining questions in the same exam, and comparing the performances of individual students on the question pairs. We find that students perform substantially and significantly better on the multiple-choice questions. In the light of this finding we reconsider the question: when students cannot correctly describe the purpose of a small piece of code, is it because they do not understand the code; because they understand its detail but are unable to abstract that detail to determine the purpose; or because they understand the purpose but are unable to express it?