DeepDect: A Facial Deepfake Video Detection Application using Ensemble Learning
Wan Xin Tay, Hui Na Chua, Muhammed Basheer Jasser, Bayan Issa, Richard T.K. Wong · 2024
The emerging technology of deepfake video poses significant threats to information integrity and public trust. Deepfake videos come in various forms, including face swaps, lip-syncing, and full-body simulations. Detection algorithms may struggle to generalize across these different types of deepfakes, leading to lower effectiveness in real-world applications. An advanced deepfake detection system is developed using an ensemble approach, combining the strengths of three pre-trained Convolutional Neural Networks (CNNs)ResNeXt50, Xception, and EfficientNet-v2-with Long ShortTerm Memory (LSTM) layers to capture both spatial and temporal features from videos to improve detection accuracy. The ensemble model is further enhanced by incorporating additional features such as lip movements, eye blinks, head poses, and color contrasts to identify subtle inconsistencies in deepfake videos that are often ignored by traditional detection methods. The ensemble model achieves an accuracy of ${8 4. 0 6 \%}$, with a good balance between precision and recall, indicating its effectiveness in real-world applications. We developed DeepDect, a web application embedded with the ensemble model, to provide a user-friendly interface for deepfake detection by uploading videos or submitting URL links. The application includes educational content about deepfake techniques to raise user awareness about the potential dangers of deepfakes.