Multimodal Misinformation Mitigation using Convolutional and Knowledge Transfer Learning Approach
Nachiket A Rathod, P. L. Ramteke · 2024
In the era of information proliferation and digital media, the battle against the spread of misinformation has evolved into a critical challenge. This work presents an inclusive exploration of the pinnacle in misinformation mitigation, employing a multimodal method that integrates textual and visual data. Advanced techniques in knowledge transfer learning are used to develop and evaluate models for Misinformation detection based on textual content, images, and their fusion. DistilFND, a distillation of diverse models specialized in different modalities, is a focal point of proposed research. The paper involves a systematic analysis of training and testing these models on a shared dataset, enabling a comparison of their performance and the provision of insightful evaluations. Cohen's Kappa scores and ROC-AUC scores are also computed to gauge the models' efficacy. This extensive study marks a significant advance in misinformation mitigation, with knowledge transfer learning and multimodal fusion serving as pivotal elements of progress. By means of a sequence of detailed experiments and evaluation metrics, this work enhances to an improved understanding of how knowledge transfer learning, multimodal fusion, and pinnacle models can boost the correctness of Misinformation detection. The results and insights in this work aid the ongoing efforts to combat the detrimental impacts of misinformation in the digital sphere.