StereoKG: Data-Driven Knowledge Graph Construction For Cultural Knowledge and Stereotypes
Awantee Deshpande, Dana Ruiter, Marius Mosbach, Dietrich Klakow · 2022
Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models.However, many techniques rely on human-compiled lists of bias terms, which are expensive to create and are limited in coverage.In this study, we present a fully datadriven pipeline for generating a knowledge graph (KG) of cultural knowledge and stereotypes.Our resulting KG covers 5 religious groups and 5 nationalities and can easily be extended to include more entities.Our human evaluation shows that the majority (59.2%) of non-singleton entries are coherent and complete stereotypes.We further show that performing intermediate masked language model training on the verbalized KG leads to a higher level of cultural awareness in the model and has the potential to increase classification performance on knowledge-crucial samples on a related task, i.e., hate speech detection.