Speech Emotion Recognition and Serious Games: An Entertaining Approach for Crowdsourcing Annotated Samples
Lazaros Matsouliadis, Eleni Siamtanidou, Nikolaos Vryzas, Charalampos A. Dimoulas · Information · 2025
Computer games have emerged as valuable tools for education and training. In particular, serious games, which combine learning with entertainment, offer unique potential for engaging users and enhancing knowledge acquisition. This paper presents a case study on the design, development, and evaluation of two serious games, “Silent Kingdom” and “Job Interview Simulator”, created using Unreal Engine 5 and incorporating speech emotion recognition (SER) technology. Through a systematic analysis of the existing research in SER and game development, these games were designed to elicit a wide range of emotion responses from player and collect voice data for the enhancement of SER models. By evaluating player engagement, emotional expression, and overall user experience, this study investigates the effectiveness of serious games in collecting speech data and creating more immersive player experiences. The research also explores the technical limitations of SER integration within game environments in real-time, as well as its impact on player enjoyment. Although there are some technology limitations due to the latency provided for real-time SER analysis, the results reveal that a properly developed game with integrated SER technology could become a more engaging and efficient tool for crowdsourcing speech data.