Face Recognition Vendor Test (FRVT) :
M Ngan, P Grother · 2014
Key ResultsCore Accuracy and Speed: Age estimation accuracy depends strongly on the provider of the core technology.Broadly, there is a twofold difference between the most accurate and the least accurate algorithm in terms of the percentage of images correctly classified to within five years and mean absolute error (MAE) 1 .Using the most accurate age estimation algorithm, (i.e., B31D from Cognitec), the chance of accurately estimating the age of a person within five years of their actual age over an ethnically-homogeneous database of 6 million images is 67%, with an MAE of 4.3 years.All algorithms can perform age estimation on a single image in less than 0.15 seconds with one server-class processor.The most accurate algorithm, on average, performs estimation in 0.125 seconds.The main dataset used for overall accuracy assessment is comprised of 6 million ethnically-homogeneous images.Although image collection was subject to the guidelines published by the Department of State (DoS), the images are compressed JPEG files which exhibit artifacts of JPEG compression causing reduction in image detail.With more detail available in less compressed images, age estimation performance may improve, but errors will still likely exist due to ageing variation driven by intrinsic and extrinsic factors. Impact of Demographic Data on Accuracy:For a heterogeneous dataset of 240 thousand images, it is empirically observed that age is more accurately estimated in males than females, with the tendency for adult females to be underestimated in age.A majority of the algorithms demonstrated lower accuracy and higher MAE on an ethnically-heterogeneous population than a homogeneous population, which suggests that ethnicity has an impact on age estimation.South Americans tend to be overestimated in age, and Asians tend to be understimated.A majority of the algorithms estimate age more accurately for the most operationally relevant age group, i.e., adults age 18-55.The adult age group is also where estimation accuracy is closest among the algorithms.The majority of algorithms exhibit the highest MAE in the senior age group, i.e., age 56-99.These results state empirical observations for the particular dataset, but they do not determine cause.The impact of extrinsic factors potentially driving the observed results between gender and ethnicity, such as cosmetics and plastic surgery, are not studied in this report.Further research would be required to objectively verify these conjectures.Age Verification Accuracy: For a system with an objective to verify that a person is at least 21 years old, a 17 year-old 1 For more details on cumulative score and mean absolute error, see sections 2.4.2 and 2.4.1.