Dual-Branch Neural Network for Bridging Semantic Gap in Harmful Meme Detection
Muhammad Shoib Amin, Summaira Jabeen, Ahmad A. Alzahrani, Abdolraheem Khader, Ali Ahmed, Sadaf Hussain · IEEE Access · 2025
Memes, cultural entities that spread predominantly through social media, have recently seen an influx of harmful content used for trolling and bullying. Detecting such harmful memes presents a significant challenge due to the semantic disparity between their textual and visual components. To address this issue, we propose a Dual-Branch Neural Network (DBNN) for bridging the semantic gap in harmful meme detection. The proposed DBNN framework utilizes global visual and textual embeddings to learn discriminative joint representations. By capturing the context between the textual and visual modalities, DBNN bridges the semantic gap, enabling robust detection of harmful content.We evaluate the performance of the proposed DBNN framework on the Harm-C and Harm-P benchmark datasets, employing Accuracy and Macro-Averaged Mean Absolute Error (MMAE) as evaluation metrics. Our experiments demonstrate that DBNN significantly outperforms state-of-the-art approaches, achieving higher accuracy and robustness in detecting harmful memes. This work highlights the potential of DBNN in effectively identifying harmful memes, contributing to a safer online environment.