Performance Comparison of Visual and Thermal Signatures for Face Recognition
Jaeyoung Heo, Besma R. Abidi, Seong G. Kong, M.A. Abidi · 2003
Face recognition is a rapidly growing research area due to increasing demands for security in commercial and law enforcement applications. Face recognition systems have reached a significant level of maturity with some practical success. However, face recognition still remains a challenging problem due to large variation in face images. The performance of face recognition systems varies significantly according to the environment where face images are taken and according to the way user-defined parameters are adjusted in several applications. Recognition based only on the visual spectrum remains limited in uncontrolled operating environments such as outdoor situations and low illumination conditions. Visual face recognition also has difficulty in detecting disguised faces, which is critical for high-end security applications. The thermal infrared (IR) spectrum comprises mid-wave infrared (MWIR) (3-5µm), longwave infrared (LWIR) (8-12µm), and short-wave infrared (SWIR) (0.9-1.7µm); all longer than the visible spectrum (0.4-0.7µm). Thermal IR imagery is independent of ambient lighting since thermal IR sensors only measure the heat emitted by objects. The use of thermal imagery has great advantages in poor illumination conditions, where visual face recognition systems often fail. This study compares the performance of recognizing the same individuals using visual and thermal infrared signatures under different operating conditions. FaceIt®, a face recognition commercial software package highly ranked by the face recognition vendor test (FRVT), is used in this study for the comparison of recognition rates using visual and thermal face images. FaceIt is based on the local feature analysis (LFA) algorithm and represents facial images in terms of local features derived statistically from a representative ensemble of faces. We conduct a detailed evaluation of this face recognition algorithm for visual and thermal images in terms of factors that include illumination, age, pose, expression, and face size. The database used in this study consists of co-registered visual and LWIR images of 3,244 (1,622 per modality) face images from 90 individuals. Research conducted by the Equinox Company used this database to show that thermal face recognition using linear discriminant analysis (LDA) yielded good performances. The evaluation is conducted using the FaceIt Identification software, which provides ranks and confidence rates of the candidates that best match an unknown person presented to the database. FaceIt uses the eye locations to normalize faces with regard to scale and orientation. Automatic eye localization via FaceIt succeeded in 95% of the cases while the remaining 5% of the images had to be aligned manually to achieve the best recognition rates for visual images. Thermal image alignment was performed in two ways and evaluations conducted separately for each case; (1) manually with the user clicking on the eye locations, and then (2) automatically by using the eye coordinates found in the registered visual images. In cases where no glasses were worn, thermal face recognition achieved higher performance ranks and confidence rates than visual face recognition under different lighting conditions and expressions. Thermal images also easily converged within 10 ranks while visual images did not in most cases. Thermal images of individuals wearing glasses resulted in poor performance since glasses block infrared emissions. The eyes in thermal images are not as obvious as in visual images. The problem of eye detection and glasses region removal is also discussed in order to achieve fully automatic face recognition in the visual and thermal infrared spectra.