Blockwise Spectral Analysis for Deepfake Detection in High-fidelity Videos
He Huang, Nan Sun, Xufeng Lin · 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA) · 2022
Deepfakes have gained widespread attention as they may give rise to a series of risks ranging from personal reputation damages to national security breaches. A mainstream approach to generating deepfakes is based on Generative Adversarial Networks (GAN). Various methods have been proposed to detect GAN-generated fake content. However, most of them only target a specific GAN and do not generalize well to other unseen GAN architectures. Moreover, many existing methods show poor performance on deepfakes that are imperceptible to the human eye as they heavily rely on visual artifacts, such as unblinking eyes and asymmetric faces. In this work, we exploit the spectral artifacts left by up-sampling operations that are universally used in GAN architectures for detecting high-fidelity deepfake videos. We first divide video frames into blocks containing the most informative areas, e.g., face, eyes, and mouth areas, and use their spectrum to train a ResNet-based classifier to detect GAN-generated images. Experimental results on public datasets show that our method is effective in detecting high-fidelity deepfakes and generalizes well across GANs with the same or similar up-sampling operations.