Head Pose and Expression Transfer Using Facial Status Score
Tomoki Hosoi · 2017
We propose a method to transfer both head poseand facial expression of a source person in a video to the faceof a target person in an output video. Our method models theentire 2D frame instead of the 3D face, and it generates outputresults using a status score, which includes the relative facialstatus about the head pose and expression in a frame. From thetarget video, the learning process obtains frame features neededfor moving to each frame from the neutral frame for all frames,and generates the basis of these features via principal componentanalysis (PCA). Then, it learns to generate these features from agiven status score sequentially. In the transfer process, it obtainsa status score from a source frame of the video and generatesthe features from the given status score. Then, it generates theoutput frame using the reconstructed features. An output videois generated by repeating these steps for each source frame.Our method generates output results on the trajectory of thetarget video by using the advantage of PCA. Therefore, in theoutput results generated by our methods, both head pose andexpression are transferred correctly while the non-face regionsof the frames are supported. Finally, we experimentally comparethe effectiveness of our method and conventional methods.