Real-time Lip Synchronization Based on Hidden Markov Models
Ying Huang, Stephen Ching-Feng Lin, Xiaoqing Ding, Baining Guo, Harry Shum · Asian Conference on Computer Vision · 2017
We propose a novel method of lip synchronization by re-using training video as much as possible when an input voice is similar to training voice sequences. Initially, face sequences are clustered from video segments, then by making use of sub-sequence Hidden Markov Models, we build a correlation between speech signals and face shape sequences. From this re-use of video, we can decrease the discontinuity between two consecutive output faces and obtain accurate and realistic synthesized animations. Our method can …