STILE: Exploring and Debugging Social Biases in Pre-trained Text Representations
Samia Kabir, Lixiang Li, Tianyi Zhang · 2024
The recent success of Natural Language Processing (NLP) relies heavily on pre-trained text representations such as word embeddings. However, pre-trained text representations may exhibit social biases and stereotypes, e.g., disproportionately associating gender with occupations. Though prior work presented various bias detection algorithms, they are limited to pre-defined biases and lack effective interaction support. In this work, we propose Stile, an interactive system that supports mixed-initiative bias discovery and debugging in pre-trained text representations. Stile provides users the flexibility to interactively define and customize biases to detect based on their interests. Furthermore, it provides a bird’s-eye view of detected biases in a Chord diagram and allows users to dive into the training data to investigate how a bias was developed. Our lab study and expert review confirm the usefulness and usability of Stile as an effective aid in identifying and understanding biases in pre-trained text representations.