ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information
Zheng Hui, Zhaoxiao Guo, Hang Zhao, Juanyong Duan, Congrui Huang · 2024
In different NLP tasks, detecting harmful content is crucial for online environments, especially with the growing influence of social media.However, previous research has two main issues: 1) a lack of data in low-resource settings, and 2) inconsistent definitions and criteria for judging harmful content, requiring classification models to be robust to spurious features and diverse.We propose ToxiCraft, a novel framework for synthesizing datasets of harmful information to address these weaknesses.With only a small amount of seed data, our framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information.Experimentation across various datasets showcases a notable enhancement in detection model robustness and adaptability, surpassing or close to the gold labels.Dataset is available at this GitHub repository.