Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning
Xiangjue Dong, Ziwei Zhu, Zhuoer Wang, Maria Teleki, James Caverlee · 2023
Pre-trained Language Models are widely used in many important real-world applications.However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in downstream applications.To address this challenge, we propose Co 2 PT, an efficient and effective debiaswhile-prompt tuning method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks.Our experiments conducted on three extrinsic bias benchmarks demonstrate the effectiveness of Co 2 PT on bias mitigation during the prompt tuning process and its adaptability to existing upstream debiased language models.These findings indicate the strength of Co 2 PT and provide promising avenues for further enhancement in bias mitigation on downstream tasks.