ConvNext-PNet: An interpretable and explainable deep-learning model for deepfakes detection
Hafsa Ilyas, Ali Javed, Khalid Mahmood Malik · 2024
The evolution of artificial intelligence (AI) techniques in recent years has increased the generation of fake content including AI-generated text, images, audio, and videos. Among which the fake visual content commonly known as deepfakes has imposed a great threat to society due to its negative impacts. To mitigate the adverse aspects of deepfakes, the research community has introduced various deepfakes detection methods. However, these deepfakes detection methods lack the interpretability and explainability of the decision-making process. The interpretable model increases trustworthiness as it provides the reasoning for classifying outcomes as real or fake. Therefore, in this paper, we have introduced ConvNext-PNet, which is a prototypical-based learning framework for the interpretable and explainable detection of visual deepfakes. In the proposed framework, prototype learning is incorporated into the modified ConvNext model that improves the discriminative features learning capability of the proposed framework along with the explainability aspect. The performance of ConvNext-PNet is evaluated on challenging datasets including FaceForensics++ (FF++), CelebDF, DFDC-P, and DeepFakeFace (DFF) datasets. The robustness of the proposed model is validated through various experiments along with the interpretability analysis. The quantitative results demonstrate the effectiveness of the model for the detection of visual manipulation, whereas the model interpretability and explainability aspect increases the trustworthiness via providing reasoning for the model predictions.