Speech Emotion Recognition with Explainable AI: Enhancing Transparency in Emotion Recognition Systems
Sejal Sharma, Aksh Dhingra, Nirbhay Bhanot, Seema Kharb · 2025
This paper explores a novel approach to Speech Emotion Recognition (SER) enhanced with Explainable Artificial Intelligence (XAI) techniques. Combining speech-based features with XAI provides interpretable insights into emotion recognition tasks. The proposed system leverages the Toronto Emotional Speech Set (TESS) dataset and employs advanced feature extraction methods, including MFCC, CQT, and RASTA. The system achieves an accuracy of 87.30%, precision of 82.25%, recall of 81.39%, and F1-score of 81.33% across 7 emotion categories. Explainable insights, provided by Local Interpretable Model-agnostic Explanations (LIME), highlight the importance of spectral and temporal features in emotion classification. This integration of XAI into SER demonstrates significant improvement, with a 10% increase in accuracy over baseline methods, making the system suitable for critical applications in telehealth, public safety, and education.