Estimating the benefits of student model improvements on a substantive scale.
Michael V. Yudelson, Kenneth R. Koedinger · 2013
Educational Data Mining researchers use various prediction metrics for model selection. Often the improvements one model makes over another, while statistically reliable, seem small. The field has been lacking a metric that informs us on how much practical impact a model improvement may have on student learning efficiency and outcomes. We pro-pose a metric that indicates how much wasted practice can be avoided (increasing efficiency) and extra practice would be added (increasing outcomes) by using a more accurate model. We show that learning can be improved by 15-22% when using machine-discovered skill model improvements across four datasets and by 7-11 % by adding individual stu-dent estimates to Bayesian Knowledge Tracing. 1.