Audio-driven human body motion analysis and synthesis
Ferda Ofli, Cristian Canton-Ferrer, Joëlle Tilmanne, Y. Demir, Elif Bozkurt, Y. Yemez, Engin Erzin, Ahmet Murat Tekalp · IEEE International Conference on Acoustics Speech and Signal Processing · 2008
This paper presents a framework for audio-driven human body motion analysis and synthesis. We address the problem in the context of a dance performance, where gestures and movements of the dancer are mainly driven by a musical piece and characterized by the repetition of a set of dance figures. The system is trained in a supervised manner using the multiview video recordings of the dancer. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Audio is analyzed to extract beat and tempo information. The joint analysis of audio and motion features provides a correlation model that is then used to animate a dancing avatar when driven with any musical piece of the same genre. Results are provided showing the effectiveness of the proposed algorithm.