A framework for bias-aware dataset evaluation in soft facial attribute recognition
Lucia Cascone, Michele Nappi, Chiara Pero, Xinggang Wang · Pattern Recognition · 2025
Soft Facial Attribute Recognition (FAR) remains largely unexplored in terms of demographic fairness. To the best of our knowledge, this study presents one of the first comprehensive analyses of demographic bias in FAR, proposing a systematic framework to detect, quantify, and promote awareness of both representational and stereotypical biases, supporting their mitigation. Leveraging established taxonomies, we evaluate state-of-the-art datasets using a rigorous set of interpretable bias metrics to uncover hidden demographic imbalances. To support reliable fairness assessment, we first enrich the datasets with standardized demographic annotations using the FairFace model. We then address label inconsistencies through the integration of predictions from advanced Vision-Language Models (VLMs). Our analysis reveals substantial imbalances across gender, age, and racial categories-specifically White, Black, and Asian- affecting dataset composition. Furthermore, we show that conventional fairness metrics often yield divergent assessments, highlighting the importance of multi-metric evaluation. This study provides a replicable methodology and actionable insights to support bias-aware facial analysis.