Pilot Study of a Tutorial Dialogue System That Emulates the Contingent Scaffolding of Human Tutors
Sandra Katz · Proceedings of the 2019 AERA Annual Meeting · 2019
This paper describes an initial classroom evaluation of Rimac, a natural-language tutoring system for physics.Rimac uses a student model to guide decisions about what content to discuss next during reflective dialogues that the student and automated tutor engage in after students solve quantitative physics problems, and how much support to provide during these discussions-that is, domain contingent scaffolding and instructional contingent scaffolding, respectively (e.g., Wood, 2001).The pilot study compared learning gains of high school students who were randomly assigned to use an experimental version of Rimac, which uses students' responses to pretest items to initialize the student model and dynamically updates the model based on students' responses to the automated tutor's questions during reflective dialogues (Chounta et al., 2017), with a control version of Rimac, which initializes its student model based on students' pretest performance but does not update the model further.Preliminary results do not reveal a significant difference in learning between conditions.Analyses of student learning gains of selected knowledge components (KCs) to which students were frequently exposed showed that the experimental condition outperformed the control.This highlights the importance of frequent exposure to KCs that students have trouble with, during the dialogues.These analyses also indicate the need for improvements to student model accuracy in the experimental version of the tutor, so that the model more reliably predicts which KCs need to be discussed and the level of granularity at which to address these KCs.