A Comparison of Error Metrics for Learning Model Parameters in Bayesian Knowledge Tracing
Asif Dhanani, Seung Yeon Lee, Phitchaya Mangpo Phothilimthana, Zachary A. Pardos · 2014
In the knowledge-tracing model, error metrics are used to guide parameter estimation towards values that accurately represent students ’ dynamic cognitive state. We compare several metrics, including log-likelihood (LL), RMSE, and AUC, to evaluate which metric is most suited for this pur-pose. In order to examine the effectiveness of using each metric, we measure the correlations between the values cal-culated by each and the distances from the corresponding points to the ground truth. Additionally, we examine how each metric compares to the others. Our findings show that RMSE is significantly better than LL and AUC. With more knowledge of effective error metrics for learning parameters in the knowledge-tracing model, we hope that better param-eter searching algorithms can be created. 1.