Unsupervised Deep Comparison for Face Anomaly Appraisal
Abdullah Hayajneh, Erchin Serpedin, Mitchell A. Stotland · 2024
This paper proposes an innovative anomaly appraisal framework that combines machine learning and signal processing for the consistent detection, localization, and evaluation of anomalies in human faces. The primary objective of this framework is to create a universal and objective metric for evaluating the degree of facial anomaly and reconstructive surgical outcomes. This metric should be well aligned with the human assessments. To accomplish this, the framework leverages the StyleGAN2 facial generator to normalize human faces that may exhibit diverse deformities. The proposed method utilizes a pre-trained Convolutional Neural Network (CNN) to extract and compare deep anomalous features between the original image and its normalized counterpart, in an unweighted manner. The resulting anomaly maps are merged into a heatmap, effectively highlighting the abnormal facial regions. This heatmap is then employed to generate a machine score, quantifying the degree of anomaly in the face. In order to assess the effectiveness of the proposed method, a comprehensive comparison of the generated anomaly maps is conducted using metrics such as Learned Perceptual Image Patch Similarity (LPIPS), Structural Similarity Index (SSIM), Pixelwise Subtraction (PS), alongside the newly introduced framework. The conducted comparative analysis demonstrates the framework's robust performance, showing a high linear correlation (0.92 Pearson's r score) and a strong monotonic relationship (0.85 Spearman's$p$score) between the human and machine generated scores. These results corroborate the framework's efficacy and its close alignment with the human judgment in assessing different levels of facial anomalies.