Spatio-temporal graph interaction networks for teacher behavior description in classroom scene
Yu Xiong, Chengyang He, Lulu Chen, Ting Cai · Engineering Applications of Artificial Intelligence · 2025
Teacher behavior description is crucial for improving teaching effectiveness and understanding classroom dynamics. Traditional methods rely on meticulous manual observation and recording, which is extremely complex and time-consuming when dealing with many teaching videos. Recently proposed video captioning technologies can automatically analyze and describe object behaviors, reducing manual intervention and providing powerful support for improving teaching effectiveness. Existing video captioning methods, however, fall short in describing teachers’ behaviors because they overlook the temporal progression of teachers’ actions, the background changes between frames, and the frequent interactions between teachers and students in real classroom environments. To address these issues, we propose a video captioning method leveraging a spatio-temporal graph interaction network (STGIN) to describe teachers’ behaviors in classroom scenes. Comprising the teacher–student spatial interaction module (TSI), teacher temporal context module (TTC), and description generator, STGIN captures spatial interaction relationships between teacher and students, extracts temporal dynamics features of the teachers’ behaviors, and precisely generates teacher behavior descriptions. To validate the proposed scheme, we collected a teacher behavioral description (TBD) dataset consisting of 2000 videos. Extensive experiments on our TBD dataset and two generic scene datasets confirm the effectiveness and robustness of the proposed STGIN in real classroom environments.