Realistic Face Reconstruction From Uncalibrated Images
Richard Roussel, André Gagalowicz · Vision Modeling and Visualization · 2004
Abstract The realistic face reconstruction described in thispaper requires only a small set of uncalibrated im-ages showing the same face and the use of a genericmodel, that is deformed in two steps. In the firststep, the deformation of the generic model is per-formed using 38 characteristic points that are re-lated to the face anatomy. An iterative techniqueinvolving camera calibration and 3D computationof the face characteristic points is performed to findthe real 3D location of each point on the specificface. A Radial Basis Function (RBF) techniquepropagates the deformation resulting from the dis-placement of the characteristic points on all the ver-tices of the model. In the second step, the silhou-ettes of the 3D face model are, in turn, matched tothose of the face images. The additional control ofthe silhouette curves gives much more informationthat allows more faithfull reconstruction of faces. 1 Introduction One of the greatest challenges in the fields of audio-visual post-production and special effects genera-tion is the handling of real objects in a filmed scene.If we want to manipulate them, they should belocated precisely. Previously manual, this searchor tracking is automated more and more. Severaltracking techniques of rigid and deformable objectsexist. Some are based only on information result-ing from images, such as 2D contours. Others use2Dor 3D models in order to guide the tracking algo-rithms. The technique presented here applies withinthe framework of 3D model-based tracking. We ex-perienced that the more realistic and close to the en-tity to be followed the 3D model will be, the moreeffective the tracking willbe. The method describedhere is devoted to the case of human faces, whichare among the most difficult entities to track, be-cause of their deformations. At this stage, we pro-pose an algorithm which allows the creation of a 3Dmesh representing faithfully a face from an a priorigiven video sequence. To produce a 3D face modelis not a new problem in the computer graphics litter-ature. Most of the techniques do not produce realis-tic faces. Realistic reconstruction of faces has beendeveloped only in the last decade, but it still requirea geat deal of experience and manual control. Butvery few of the latter techniques construct modelsable to match precisely an a priori given set of pho-tographs or pictures of a specific face. The image-based techniques allowing to reconstruct 3D facesare either lack precision (mainly computer vision-based techniques) or require the use of 3D scannersand/or large face databases. The aim of our workis to propose a technique allowing the realistic re-construction of 3D faces but we want this techniqueto be affordable for any computer graphist (no needto use 3D scanners nor big databases). It has to beprecise but rapidely executed. The main features ofour technique are the following:1. The input isa simple setof N images of a givenface. This number N may be very small, evenequal to one.2. Interaction of the user is required, but the tech-nique is very easy to understand and use.3. 3D face is constructed in roughly 15 minutes;this time is mainly an interaction time, as com-putations are negligeable.4. The 3D face obtained projects exactly on allimages used as input.5. Use of a generic 3D face model. Any modelthat can be found in the web is suitable, so thatit is not a constraint.