A multimodal attention-based system for real-time listening comprehension evaluation and feedback
Zhitong Chen · 2025
This paper studies an intelligent evaluation and real-time feedback algorithm for English listening comprehension based on multimodal learning. This paper constructs a multi-modal data modeling framework, integrates voice, text and visual information, and accurately evaluates learners' listening comprehension ability through dynamic attention fusion mechanism and hierarchical multi-task learning framework. In addition, a real-time feedback system is designed, which uses adaptive learning algorithm and affective computing technology to provide personalized feedback according to learners' learning behavior and preferences. The experimental results show that this method is superior to the traditional method in accuracy, F1 score and AUC, and the feedback delay is significantly shortened to 220 ms. At the same time, the system can still maintain good response speed and processing ability under high concurrent requests, and the feedback adoption rate is significantly improved. Error analysis reveals four main error types and their detection accuracy, which provides a basis for subsequent optimization. This study provides new ideas and methods for the intelligent and personalized development of English listening teaching.