Enhancing Offensive Bengali Social Media Meme Detection: A Weighted Ensemble Architecture for Predicting Type and Target Classes
Md. Maruful Islam, Nidita Roy, Tania Sultana Sheila, Muhammad Ibrahim Khan · 2023
This paper investigates the classification of offensive memes using a weighted ensemble approach of multimodal models that integrates visual and textual information. A meticulously curated dataset, Multi-modal Offensive Memes Bengali Dataset (MoMBD), containing 3662 offensive Bengali memes, is created to facilitate work in this domain. A three-level hierarchical classification system is followed to accurately categorize the fine-grained categorization of offensive memes. In this paper, BERT, as text classifier, is combined with transfer learning model, as image classifier, to create a multimodal unit. Then, a weighted ensemble is performed on the three best-performing multimodal models. The proposed weighted ensemble technique is effective in achieving high accuracy and f1-scores for both coarse-grained and fine-grained classifications. The proposed method achieves an impressive weighted f1-score of 71.84% in fine-grained classification. Overall, this research contributes valuable insights and techniques for detecting offensive Bengali memes, type and their target.