Multimodal Aggressive Meme Classification using Bidirectional Encoder Representations from Transformers
Md Ashraful Alam, Jawad Hossain, Shawly Ahsan, Mohammed Moshiul Hoque · 2024
Aggressive memes, often characterized by harmful or provocative content, pose a challenge for detection across multiple modalities. To address this, we developed the MAC dataset, categorizing memes into non-aggressive, political, religious, gendered, and other forms of aggression. We evaluated various textual models, including CNN, BiLSTM, various BERT, and visual models, such as VGG16, VGGG19, ResNet50, and Vision Transformer, employing a decision fusion technique for multimodal analysis. Bangla BERT-1 achieved the highest f1-score (0.74) among textual models, while Vision Transformer led visual models (f1-score: 0.68). Our multimodal approach combining Bangla BERT and Vision Transformer outperformed individual models with a weighted f1-score of 0.76, excelling in political and religious aggression classes though underperforming in non-aggressive and gendered categories.