Learning Bias-reduced Word Embeddings Using Dictionary Definitions
Haozhe An, Xiaojiang Liu, Donald Zhang · Findings of the Association for Computational Linguistics: ACL 2022 · 2022
Pre-trained word embeddings, such as GloVe, have shown undesirable gender, racial, and religious biases.To address this problem, we propose DD-GloVe, a train-time debiasing algorithm to learn word embeddings by leveraging dictionary definitions.We introduce dictionaryguided loss functions that encourage word embeddings to be similar to their relatively neutral dictionary definition representations.Existing debiasing algorithms typically need a precompiled list of seed words to represent the bias direction, along which biased information gets removed.Producing this list involves subjective decisions and it might be difficult to obtain for some types of biases.We automate the process of finding seed words: our algorithm starts from a single pair of initial seed words and automatically finds more words whose definitions display similar attributes traits.We demonstrate the effectiveness of our approach with benchmark evaluations and empirical analyses.