Test-size Reduction for Concept Estimation
Divyanshu Vats, Christoph Studer, Andrew Lan, Lawrence Carin, Richard G. Baraniuk · 2013
Consider a large database of questions that assess the knowl-edge of learners on a range of different concepts. In this paper, we study the problem of maximizing the estimation accuracy of each learner’s knowledge about a concept while minimizing the number of questions each learner must an-swer. We refer to this problem as test-size reduction (TeSR). Using the SPARse Factor Analysis (SPARFA) framework, we propose two novel TeSR algorithms. The first algorithm is nonadaptive and uses graded responses from a prior set of learners. This algorithm is appropriate when the instruc-tor has access to only the learners ’ responses after all ques-tions have been solved. The second algorithm adaptively selects the “next best question ” for each learner based on their graded responses to date. We demonstrate the efficacy of our TeSR methods using synthetic and educational data. Keywords Learning analytics, sparse factor analysis, maximum likeli-hood estimation, adaptive and non-adaptive testing 1.