Fusion Network for Multimodal Hate Speech Detection
Phuc H. Duong, Truc Thuy Nguyen, Hien T. Nguyen · 2024
In an era where social network platforms are increasingly plagued by harmful content, effective detection methods are crucial for maintaining healthy online communities. In this paper, we introduce a multimodal approach to hate speech detection, leveraging a fusion network that integrates text and image analysis. We mainly use state-of-the-art pre-trained language models like DistilBERT, MPNet, and image processing models such as ResNet and EfficientNetV2. These models facilitate a nuanced understanding of both intra- and inter-modality content dynamics. Additionally, we compare the effectiveness of context-free word embedding models like GloVe and fastText with contextual language models. Extensive experiments on the MMHS150K dataset, a popular and specialized dataset for multimodal hate speech analysis, demonstrate competitive performance compared with existing multimodal hate speech detection methods.