The FineMotion entry to the GENEA Challenge 2023: DeepPhase for conversational gestures generation
Vladislav Korzun, Anna Valer'evna Beloborodova, Arkady Ilin · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · 2023
This paper describes FineMotion’s entry to the GENEA Challenge 2023. We explore the potential of DeepPhase embeddings by adapting neural motion controllers to conversational gesture generation. This is achieved by introducing a recurrent encoder for control features. We additionally use VQ-VAE codebook encoding of gestures to support dyadic setup. The resulting system generates stable realistic motion controllable by audio, text and interlocutor’s motion.