Spatial-Temporal Variables for Deepfake Video Identification
Yeeshu Ralhen, Sharad C. Sharma · 2023
Deepfake detection at the video level has not been as widely studied as at the image level, as it involves analyzing sequences of frames in a temporal context to detect manipulation, which is more complex than analyzing individual images. In this research, we seek to demonstrate how the current image and sequence classifiers-based approaches for deepfake detection struggle to generalize to new manipulation techniques. In order to improve the generalisation ability to identify new types of deepfake images or videos, we suggest spatio-temporal variables that are described by 3D CNNs. We demonstrate that spatial features acquire unique deep fake method-specific qualities, but spatio-temporal features capture common deep fake method-specific attributes. Using the FaceForensics++ dataset, we present a detailed examination of how the sequential and 3D CNN video encoders are making use of temporal information. We discover that, in contrast to previous sequence encoders, our method incorporates local spatio-temporal relationships as well as irregularities in the deepfake videos. We demonstrate that our technique surpasses conventional approaches in terms of generalisation abilities via substantial investigations on the FF++.