An analysis of the application of intelligent tutoring systems on students' self-regulated learning development
Chen Wang, Gerard B. Rowe, Nasser Giacaman, Cathy Gunn · 25th Annual Conference of the Australasian Association for Engineering Education : Engineering the Knowledge Economy: Collaboration, Engagement & Employability · 2014
Background: Intelligent tutoring systems (ITS) can provide one-to-one tutoring opportunities to every student at any time and in any place by applying artificial intelligence technologies to model students' learning progress and understand teachers' teaching strategies. As attractive as this may sound, the reality is that the potential of such an ITS has yet to be realized and in practice such systems have had only a limited impact on classrooms. One potential reason for the slow uptake of such a teaching approach is that students studying alone with an ITS require a higher level of self-regulated learning skills (SRL). If these skills aren't properly inculcated it is less likely that students will commit to long-term use of an ITS. Purpose: Previous studies have reported that SRL skills have significant positive impacts on student's learning outcomes. Much of the research on applying ITS to improve student's SRL abilities has accumulated from 1998 to 2014. So far, however, there has been little discussion about the available options to measure and support SRL in these types of ITS, and the system's effectiveness on learner's academic performance. Design/Method: The study collected 53 empirical studies of designing, implementing and evaluating ITS - those ITS which not only teach domain knowledge but also foster a learner's SRL abilities. Meta-analysis was applied as the main research method in order to gain comprehensive insights into the ITS in this field. Results: Our analysis suggests that no ITS in this field had negative impact on students' learning outcomes. The results of this research support the idea that ITS is an effective teaching method to foster a student's SRL abilities, and that use of such an ITS has a small positive impact on student's learning outcomes. Conclusions: The ITS evaluated here used learning analytics and collected event data to measure a learner's SRL progress, rather than using self-reported data, which might be common for traditional classroom instruction. The fundamental modes of scaffolding were prompts and feedback. Moreover, the timing of prompt's appearance (just-in-time or delayed), and the content of the feedback (context-sensitive or context-insensitive) had a significant influence on student's learning outcomes. The SRL mechanism could be embedded in an ITS from the initial design stage or be retro-fitted to an existing ITS as addon agents afterwards. Most of these systems were model-tracing ITS and multi-agent systems. While most of the ITS in this field were designed for STEM (Science Technology Engineering and Mathematics) subjects, relatively little was found in the literature on ITS applied in Engineering education, and in particular in electrical engineering.