How Well Can Tutoring Audio Be Autoclassified and Machine Explained With XAI: A Comparison of Three Types of Methods
Lishan Zhang, Linyu Deng, Sixv Zhang, Ling Chen · IEEE Transactions on Learning Technologies · 2024
With the popularity of online one-to-one tutoring, there are emerging concerns about the quality and effectiveness of this kind of tutoring. Although there are some evaluation methods available, they are heavily relied on manual coding by experts, which is too costly. Therefore, using machine learning to predict instruction quality automatically is an effective way to reduce human costs. Three classification methods are analyzed in this paper: 1) Random forest algorithm with human engineered descriptive features; 2) LSTM algorithm with acoustic features generated by openSMILE toolkit; 3) CNN algorithm with Mel spectrogram of the audio. The results show that the three approaches can complete the prediction task well, with the second approach exhibiting the best accuracy. The importance of the features in these classification models is analyzed according to eXplainable Artificial Intelligence techniques (i.e. XAI) and statistical feature analysis methods. In this way, key indicators of high-quality tutoring are identified. This study demonstrated the usefulness of XAI techniques in understanding why some tutoring sessions are of good quality and others are not. The results can be potentially used to guide the improvement of online one-to-one tutoring in the future.