Navigating the Obscured: A Novel Deepfake Detection Framework Tackling Trending Occlusions in Social Media
Bristy Biswas, Avi Deb Raha, Chandan Paul, Sumit Kumar Dam, Apurba Adhikary, Rameswar Debnath, Anupam Kumar Bairagi, Mrityunjoy Gain · 2024
Images and videos have become a ubiquitous part of modern life, used extensively for entertainment and security purposes. In recent years, Generative AI (Gen-AI) has gained immense popularity due to its advanced capabilities in generating highly realistic images. However, the use of Gen-AI in creating deepfakes—manipulated videos or images designed to mimic real individuals has also increased. Deepfakes have become a growing concern, particularly for the authenticity of visual content shared online. Detecting these fabricated visuals remains a critical challenge. While several methods exist for detecting deepfakes, new trends in image manipulation on social media have introduced significant complications. One of the recent trends involves occlusions, such as large text overlays or windows inserted into images, commonly seen on various platforms. These occlusions pose a serious challenge to current detection techniques, as they often interfere with the model’s ability to recognize facial features accurately. In this research, we address these newly emerging occlusion patterns and propose a novel deepfake detection model capable of effectively identifying fake images even in such obstructions. We propose a deep learning model that can recognize both clean and occluded deepfakes. Our approach integrates knowledge of both clean and occluded images, allowing the model to detect deepfakes under a variety of conditions, including occlusions created by text, stickers, or other overlays. We conducted an extensive study of trending occlusion patterns, focusing specifically on images featuring text within a window overlay. We analyzed how various factors such as occlusion size, shape, and position influenced model accuracy. Our model, tested on both traditional clean images and those with occlusions, demonstrated robust performance. In particular, DenseNet121 outperformed ResNet50 and VGGNet16 in handling these challenging occluded images with an accuracy of more than 99%.