Illumination Enlightened Spatial-temporal Inconsistency for Deepfake Video Detection
Kaiyue Tian, Chen Chen, Yichao Zhou, Xiyuan Hu · 2024
The rapid advancement of facial manipulation techniques has greatly simplified the creation of deepfake videos, posing a major threat to social safety, public opinions and even political stability. Existing deepfake detection methods primarily concentrate on capturing spatial artifacts or extracting uniform temporal inconsistency, neglecting the potential of exploiting dynamic spatiotemporal inconsistency. To address these issues, this paper proposes a novel network that effectively leverages dynamic spatiotemporal inconsistency, termed DSTI, by integrating the sequential illumination features and intra/inter-frame clues. The proposed DSTI contains two branches: one branch employs a transformer encoder to perform inconsistency computation from sequential illumination representations derived from 3D facial models, including illumination coefficients, 3D normal vectors, and luminance values. The other branch utilizes a timesformer network to capture intra/inter-frame inconsistency from sampled videos. Extensive experimentation validates that the proposed method outperforms other competitive approaches.