Photorealistic Face Transfer in 2D and 3D Video
Daniel Merget, Philipp Tiefenbacher, Mohammadreza Babaee, Nikola Mitov, Gerhard Rigoll · Lecture notes in computer science · 2015
3D face transfer has been employed in a wide field of settings such as videoconferencing, gaming, or Hollywood movie production. State-of-the-art algorithms often suffer from a high sensitivity to tracking errors, require manual post-processing, or are overly complex in terms of computation time. Addressing these issues, we propose a lightweight system which is capable to transfer facial features in both 2D and 3D. This is accomplished by finding a dense correspondence between a source and target face, and then performing Poisson cloning. We solve the correspondence problem efficiently by a sparse initial registration and a subsequent warping, which is refined in a surface matching step using topological projections. Additional processing power is saved by converting extrapolation problems to simple interpolation problems without loss of precision. The final results are photorealistic face transfers in either 2D or 3D between arbitrary facial video streams. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.