Multi-Model Behavior Synchronizing Prosody Model in Sign Language Synthesis
Yi Chen · Chinese Journal of Computers · 2006
This paper proposes a multi-model behavior synchronizing prosody model and its application to Chinese sign language synthesis. Based on huge realistic multi-model behavior training data, the authors adopt learning the prosody mode for each single channel behavior data and further synchronizing relation model of all models, and present the framework for the multi-model synchronization in virtual human synthesis, including models of sign language, speech, facial expression and lip movement and so on. The formal description of multi-model prosody model is demonstrated in detail. Comparing to traditional regularity approaches, the learning based approach in this paper is more adequate to express complicatedly the multi-model synchronizing relationship, and to synthesize realistically the multi-model behavior of the virtual human. As the example, the synchronizing prosody model involving sign language prosody parameters and speech prosody parameters is given. The authors design an approach to compute the prosody parameters and apply it to control the virtual human's multi-model behavior synchronously. Experiments based on the Coss (863 speech material library) and Chinese sign language library show that the multi-model behavior synchronizing prosody model works well. It enhances the recognition rate of synthetic sign language by 5.94%.