Developing an explainable human action recognition system for academic environments: Enhancing educational interaction
Mustafa Basthikodi, Poornima B-V · Results in Engineering · 2025
• Explainable Action Recognition for Education : Enhances educational interactions in academia. • Temporal Segment Network (TSN) : Captures dynamic temporal patterns for robust action recognition. • Segmented Video Processing : Analyzes video segments independently for comprehensive action insights. • Improved Communication for Non-Verbal Cues : Valuable for individuals with communication challenges, highlighting gestures. • High Accuracy and Minimal Loss: achieves a high recognition accuracy of 98.6 % with minimal loss. This study presents the development of an Explainable Human Action Recognition System tailored for academic environments, aiming to enhance educational interactions. By leveraging a Temporal Segment Network as the core methodology, the system achieves robust action recognition through a multi-segment approach that captures dynamic temporal patterns in video data. The TSN divides video sequences into several segments, processes them individually, and integrates the information to provide a comprehensive understanding of the observed actions. This allows the system to discern and classify a wide range of educational activities, from classroom gestures to interactive learning behaviors. A critical feature of the proposed system is its explainability, ensuring that the recognized actions and the underlying decision-making processes are transparent and interpretable to educators and learners alike. Human action recognition is particularly valuable in contexts where non-verbal communication is prevalent, such as in interactions involving individuals with communication challenges, where gestures play a crucial role in conveying information and emotions. Our results demonstrate that this approach not only improves the accuracy of action recognition, achieving a 98.6 % accuracy with minimal loss, but also supports the detailed analysis of educational interactions, ultimately contributing to more engaging and effective learning experiences.