Understanding Sensitive Attribute Association Bias in Recommendation Embedding Algorithms
Lex Beattie, Isabel Corpus, Lucy H. Lin, Praveen Ravichandran · ACM Transactions on Recommender Systems · 2025
We present a novel evaluation framework for assessing representation bias in recommendation embedding algorithms and demonstrate its utility through two case studies. Attribute association bias (AAB), defined as the semantic capture of sensitive attributes in the trained recommendation latent embedding space, is a risk in industry recommendation settings due to the popularity of embedding-based recommendation algorithms and the common practice of leveraging embedding outputs in hybrid recommendation systems; biases in the representation space could reinforce harmful stereotypes and propagate representational harms. We introduce the three-step ESA framework for characterizing the e xistence, s ignificance, & a mplification of AAB in these types of algorithms, and provide practical methods inspired by natural language processing (NLP) research on gender bias to implement each of these steps. We demonstrate the utility of our framework with an industry case study of a production-level podcast recommendation model for downstream users. We uncover significant levels of user gender AAB when user gender is both used and removed as a model feature during training, pointing to the potential for systematic bias in recommendation embedding model outputs. Additionally, we showcase the versatility of the ESA framework by implementing it to analyze user gender AAB in previously published MovieLens-1M recommendation entity embeddings.