Investigating the Robustness of Knowledge Tracing Models in the Presence of Student Conceptual Drift
Morgan P Lee, Artem Frenk, Karish Gupta, Thinh Hung Pham, Ethan A. Croteau, Neil Thomas Heffernan · 2024
Knowledge Tracing (KT) has been an established problem in the educational data mining field for decades,and it is commonly assumed that the underlying learning process being modeled remains static. Giventhe ever changing landscape of online learning platforms (OLPs), we investigate how the constructs ofconcept drift and code decay can impact student behavior within an OLP through testing model perfor-mance both within a single academic year and across multiple academic years. Four well-studied KTmodels were applied to five academic years of data to assess how susceptible KT models are to conceptdrift. Through our analysis, we find that all four families of KT models can lose accuracy with time,and that the relationship between model complexity and susceptability to concept drift is not as simple aspreviously theorized. Code used to conduct our analyses is available at https://github.com/ASSISTments-IQP/LongitudinalKnowledgeTracing24/tree/master, and the data at https://osf.io/hvfn9/.