Cyberbullying Detection in Social Media with Multimodal Data Using Transfer Learning
C. Valliyammai, S. Monish Raaj, B.L. Athish, J. Kishore Kumar · 2024
Cyberbullying persists as a prevalent issue in today's digital landscape, posing significant challenges to individuals' well-being. A comprehensive system for detecting and categorizing cyberbullying instances in online content is proposed, leveraging multimodal data integration of textual and visual cues to discern the presence and nature of such behavior. Subsequent preprocessing employs Optical Character Recognition (OCR) for text extraction from images, alongside advanced cleaning and tokenization techniques for both text and images. Feature extraction utilizes cutting-edge methods such as Word2vec and emotion2vec for textual data, and ResNet50 for automatic feature extraction in visual data. A model fusion stage unifies textual and visual information through self-attention mechanisms and Contrastive Language-Image Pre-training (CLIP), enriching feature interaction for deeper data understanding. Training phases employ language specific transformers like Bidirectional Encoder Representations from Transformers (BERT) is used for textual sentiment analysis, and Robustly Optimized BERT Approach (RoBERTa) is used for sentiment analysis in the images. The visual features and emotion are identified using DenseNet and Vision Transformer (ViT). The fully connected layer consolidates information to categorize content into bullying and non-bullying classes, with further subdivisions into categories such as age, ethnicity, gender, and religion. The proposed system not only offers a robust multimodal strategy for cyberbullying detection but also offers nuanced classification, facilitating targeted intervention which creates safer online environments.