Facial Expression Recognition Based on
Theekapun Charoenpong · Fukui University Repository (University of Fukui) · 2008
Due to a problem of current research occurring when recognizing facial expressions \tfrom a 2.5D partial face data set taken from any viewpoint ranging from -45 degrees to +45 \tdegrees, we propose a new effective method for recognizing facial expressions based on \t3D face reconstruction. This proposed method consists of two parts. In the first part, a 3D \tface is reconstructed by using an ellipse fitting technique. A face vector is used to detect a \tsymmetry plane. By using the symmetry plane and real face data, the 3D face can be \treconstructed. In the second part, facial expression is represented in terms of the change of \tcrossing points on a face plane. Two crossing point analysis methods consisting of a \tcrossing point distribution method and a displacement vector method are used to analyze \tdistribution of crossing points. By using a support vector machines method, facial \texpressions can be recognized. Four facial expressions which were used in experiments \twere neutral, anger, surprise and smiling. The results show the feasibility of the proposed \tmethod. \tIn the chapter 2, a 3D face is reconstructed. The data from each cross section of the \thead is fitted by an ellipse fitting technique. In each cross section of the head data, a vector \tpointing from a center of the ellipse to the facial surface is defined as a semi-major axis \tvector. Based on the experiments of all viewpoints, each cross section shows the different \taverage ellipse fitting error. According to this property, the semi-major axis vector under \tthe condition that the average ellipse fitting error of its cross section is less than 1 mm tend \tto point to the surface near the nose. By averaging the semi-major axis vectors obtained \tfrom one viewpoint, a face vector can be calculated. A nose ridge detection algorithm is \tused to detect the nose tip and the nose ridge. Vectors which are the projection of the face \tvector in the cross sections in the nose region are defined as projected face vector. Points \ton the facial surface in each cross section are projected onto the projected face vector. The \tprojected points from the nose tip and nose ridge always show farthest distance from the \tcenter of the ellipse. To detect the nose tip and the nose ridge, the projected face vector is \tcorrected to the point which shown the farthest distance. By using the nose tip and the nose \tridge points, two vectors are obtained. One is a nose ridge vector. This vector is computed \tby a Principle Components Analysis method (PCA). Another one is a nose vector. The nose \t2 \tvector points from the center of the head to the average of the nose ridge point. By using \tthe two vectors, a symmetry plane is defined. This symmetry plane is corrected by using a \tsymmetry plane correction algorithm. The 3D face is then reconstructed. \tIn the chapter 3, a facial expression is represented in a face plane. From a \treconstructed 3D face, facial surface is extracted by using a L*a*b* color space. A face \tplane algorithm is used to represent a facial expression in terms of crossing points. A face \tplane is a virtual plane across the head. Four neighboring points, which are points on the \tfacial surface, are used to compute a normalized vector. The normalized vectors point to \tthe center of the head. By using the normalized vectors, the face plane is computed. A point \ton the face plane which a normalized vector points through is defined as crossing point. In \teach facial expression, the character of the crossing point distribution is different. The \tcrossing point distribution of an expression face is compared with a neutral face from the \tsame person for recognition. Two methods are used to analyse the crossing point \tdistribution. The first one is a crossing point distribution method. The crossing points on a \tface plane are divided into 36(6x6) sub-areas. The numbers of the crossing points in 36 \tsub-areas are used for the facial expression recognition. The second one is a displacement \tvector method. The size of an expression face is normalized to the size of a neutral face. \tPoint on the facial surface with 3D-coordinate system is then converted to 2D-coordinate \tsystem. Two points on two faces which have the same 2D-coordinate are used to define a \tdisplacement vector. A displacement vector is a vector which points from a crossing point \tof a neutral face to a crossing point of an expression face. The two crossing points \tcorrespond to the pair point on the two faces. A surface on the expression face is divided to \tinto 36(6x6) sub-areas based on the size of the neutral face. Displacement vectors in a \tsub-area are represented by an average of the displacement vectors. The 36 average \tdisplacement vectors are used for facial expression recognition. \tIn the chapter 4, experiments were done. The head pose variation in yaw and two \tsituations of the head – with and without a cap were used. There were three experiments \tdone. First, the nose ridge point was detected from a 2.5D partial face data set taken from \tany viewpoint ranging from -80 degrees to +80 degrees. The 2.5D partial face data were \tcaptured from 27 persons. All of the nose ridge from 223 samples with the cap can be \t3 \tdetected successfully. While ten samples fail in detection. Failure occurs because of the \tinfluence of messy hair. Second, the 3D face was reconstructed from a 2.5D partial face \tdata set taken from any viewpoint ranging from -45 degrees to +45 degrees. The 2.5D \tpartial face data were captured from 22 persons with and without a cap. A criterion defined \tby using a facial recognition was used to evaluate the reconstructed 3D face. By using the \tcriterion, four samples of 187 samples were evaluated as an incorrect reconstruction. Third, \tthe facial expressions were recognized by using a support vector machines technique and \tthe radial basis function. 2.5D partial face data sets from 22 persons were taken from any \tviewpoint ranging from -45 degrees to +45 degrees. Four facial expressions which were \tsmiling, surprise, anger and neutral were used. The experiments of the facial expression \trecognition were done based on the assumption that the person is known. The average \trecognition accuracy using the crossing point distribution scheme and the displacement \tvector scheme were 62.6% and 78.9% respectively. The advantages of our proposed \tmethod is that only one 2.5D partial face data set captured from any viewpoint of the face \trotation between -45 degrees and + 45 degrees can be used. The results show the feasibility \tof the proposed method. In the chapter 5, the results of the proposed method were \tconcluded.