Evaluation of algorithms for recommending intelligent tutors to computer science students
Bharatwaj Appsamy, Christopher Scaffidi · Journal of computing sciences in colleges · 2014
Computer science students often need help with discovering and learning how to use APIs, such as when working on homework. Prior research showed that students can learn APIs faster and more effectively from intelligent tutors, compared to either textbooks or searching for examples online. Yet to date, no facility has been available to help students learn from a sequence of tutors, for the common situation where they need to learn multiple APIs. This paper describes a system that provides API tutor recommendations; it integrates two alternative algorithms that determine, for a given user at a given moment in time, what API tutors to recommend for that user's consideration. One algorithm is a content-based recommender, which builds a graph reflecting relationships among APIs (in terms of how they call one another) to determine an order in which it makes sense to learn the APIs. The other algorithm is a standard collaborative filtering algorithm, which identifies which users tend to give similar ratings as one another, then uses the ratings from users to recommend tutors to similar users. In an empirical study, 25 students used the system for two weeks, worked through recommended tutors, and gave ratings to the tutors. The study tracked which of the two algorithms gave each recommendation, making it possible to statistically determine which algorithm tended to give recommendations that obtained higher ratings. The results showed that ratings of content-based recommendations were significantly higher than ratings of collaborative-filtering-based recommendations. Further analysis of study data suggested that the reason for this difference was that the first algorithm's recommendations were more relevant and provided at points in time when the students felt more ready to learn each API.