On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang · Findings of the Association for Computational Linguistics: ACL 2022 · 2022
Dialogue safety problems severely limit the real-world deployment of neural conversational models and have attracted great research interests recently.However, dialogue safety problems remain under-defined and the corresponding dataset is scarce.We propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in humanbot dialogue settings, with focuses on contextsensitive unsafety, which is under-explored in prior works.To spur research in this direction, we compile DIASAFETY, a dataset with rich context-sensitive unsafe examples.Experiments show that existing safety guarding tools fail severely on our dataset.As a remedy, we train a dialogue safety classifier to provide a strong baseline for context-sensitive dialogue unsafety detection.With our classifier, we perform safety evaluations on popular conversational models and show that existing dialogue systems still exhibit concerning contextsensitive safety problems.