Vocal emotion identification in hearing loss : Insights from emotion expression acoustics
Mattias Ekberg · Studies in disability research · 2025
Individuals with hearing loss often face challenges in identifying emotions conveyed through voice, even when using hearing aids or cochlear implants.These difficulties can impact emotional communication and are linked to reduced accuracy in identifying vocal emotion expressions.To better support individuals with hearing loss, it is important to understand how emotion recognition in voices can be improved.This doctoral thesis comprises four studies organized around two main themes.The first theme investigates which acoustic features distinguish different emotional expressions in speech and how these features contribute to listeners' ability to identify emotions.The second theme examines how mild-to-moderate sensorineural hearing loss affects vocal emotion identification, and whether signal amplification can enhance identification accuracy.The findings show that fear is the most acoustically distinct emotion in spoken sentences, while happiness is the least distinct.Acoustic features related to frequency and spectral balance were most effective in predicting fear, whereas happiness was best predicted by a broader set of acoustic parameters.Surprise was also acoustically well-defined, particularly through amplitude and temporal features, but its identification was not strongly predicted by these parameters.Importantly, restoring audibility through linear amplification significantly improved the identification of happiness in speech, and anger, fear, and interest in nonverbal vocalizations.These emotions were characterized by distinct frequency-related features.Additionally, surprise was the most accurately identified emotion in speech by both individuals with hearing loss and those with normal hearing.However, patterns of confusion between emotions could not be explained by acoustic similarities alone.The results also suggest that temporal fine structure may play a key role in emotion identification when other cues are limited.Based on these findings, this doctoral thesis offers insights into the acoustic basis of vocal emotion expressions and proposes strategies to support emotion recognition in individuals with hearing loss.