EngaGes: An Engagement Fused Co-Speech Gesture Synthesis Model
Haipeng Lu, Nan Xie, Zhengxu Li, Wei Pang, Yongming Zhang · 2024
Co-speech gestures are an important way for humans to efficiently engage in natural communication, which is also very inspiring for human-robot interaction. How to synthesize natural and humanized co-speech gestures is crucial for designing robots and providing a humanized human-robot interaction experience. In intensive interactive scenarios (such as education), co-speech gestures are not only related to speech content, but also influenced by the level of engagement during the interaction. Few research on the synthesis of co-speech gestures pay attention to this point. We design a multi-modal fused co-speech gesture synthesis model based on existing co-speech gesture synthesis algorithms. This model can effectively fuse engagement and audio to synthesis co-speech gestures.