Label-aware Hard Negative Sampling Strategies with Momentum Contrastive Learning for Implicit Hate Speech Detection
Jaehoon Kim, Seungwan Jin, Sohyun Park, Someen Park, Kyungsik Han · 2024
Detecting implicit hate speech that is not directly hateful remains a challenge.Recent research has attempted to detect implicit hate speech by applying contrastive learning to pretrained language models such as BERT and RoBERTa, but the proposed models still do not have a significant advantage over cross-entropy loss-based learning.We found that contrastive learning based on randomly sampled batch data does not encourage the model to learn hard negative samples.In this work, we propose Label-aware Hard Negative sampling strategies (LAHN) that encourage the model to learn detailed features from hard negative samples, instead of naive negative samples in random batch, using momentum-integrated contrastive learning.LAHN outperforms the existing models for implicit hate speech detection both inand cross-datasets.