Coarse-To-Fine Key Frame Selection for Dynamic Facial Expression Recognition
Shaoqi Yan, Yan Wang, Xinji Mai, Qing Zhao, Boyang Wang, Wei He, Wenqiang Zhang · 2023
In studies of in-the-wild Dynamic Facial Expression Recognition (DFER), most studies have focused on exploring spatial-temporal features, but neglects expression peak frame (key frame) in video sequence, which is crucial for accurate expression recognition. These key frames, capturing the most intense moments of an expression, are essential for understanding the full context and nuances of facial expressions in a video. In this paper, we propose a dynamic facial expression recognition method called C2F-KFS, which extracts key frames from video sequences by extracting features from coarse to fine, in order to improve the robustness of classification results. We designed the coarse-level spatial key frames extraction (CSKFE) module to extract spatial features from different facial regions. In addition, the fine-level temporal key frames extraction (FTKFE) module to extract feature from temporal perspective and make weighted comparison, in order to select key frames. By extensive experimental on two popular in-the-wild datasets, DFEW and FERV39k, the results and visualization show that the C2F-KFS method achieves state-of-the-arts DFER. Meanwhile, we have conducted ablation experiments to prove the effectiveness of C2FKFS method.