Translation-Delay-Aware Emotional Avatar System for Online Communication Support
Tomoya Suzuki, Akihito Taya, Yoshito Tobe, Guillaume F. Lopez · 2024
Virtual avatars that can express emotions are becoming popular and widely used for video streaming, tourist information system, communication in the metaverse, and so on. By adding an automatic translation function to such avatars, they can further support communication with foreign listeners and tourists. However, since automatic translation takes some time, there’s a time lag between the speaker’s facial expression being reflection in the avatar and the translated text presentation. Such delay may disturb users’ conversations. This paper tackles the above issue by proposing an emotion reflection system for virtual avatars that considers the time delay until translated text presentation. The proposed system stores the recognized speaker’s emotion and matches the timing of emotion reflection on the avatar’s face with the timing of spoken translated read-aloud voice. Therefore, we can reduce the gap between the avatars’ faces and voice emotions.