Predictors of student satisfaction: a large-scale study of human-human online tutorial dialogues
Guanliang Chen, David Lang, Rafael Fernandes de Abreu e Lima Ferreira, Dragan Gašević · Monash University Research Portal (Monash University) · 2019
For the development of successful human-agent dialogue-based tutoring systems, it is essential to understand what makes a human-human tutorial dialogue successful. While there has been much research on dialogue-based intelligent tutoring systems, there have been comparatively fewer studies on analyzing large-scale datasets of human-human online tutoring dialogues. A critical indicator of success of a tutoring dialogue can be student satisfaction, which is the focus of the study reported in the paper. Specifically, we used a large-scale dataset, which consisted of over 15,000 tutorial dialogues generated by human tutors and students in a mobile app-based tutoring service. An extensive analysis of the dataset was performed to identify factors relevant to student satisfaction in online tutoring systems. The study also engineered a set of 325 features as input to a Gradient Tree Boosting model to predict tutoring success. Experimental results revealed that (i) in a tutorial dialogue, factors such as efforts spent by both tutors and students, utterance informativeness and tutor responsiveness were positively correlated with student satisfaction; and (ii) Gradient Tree Boosting model could effectively predict tutoring success, especially with utterances from the later period of a dialogue, but more research effort is needed to improve the prediction performance.