A multimodal face antispoofing method based on multifeature vision transformer and multirank fusion
Zuhe Li, Yuhao Cui, Fengqin Wang, Weihua Liu, Yongshuang Yang, Zeqi Yu, Bin Jiang, Hui Chen · Concurrency and Computation Practice and Experience · 2023
Summary Face antispoofing (FAS) is attracting increasing attention from researchers because of its important role in preventing facial recognition systems from face spoofing attacks. With the advancement of various new convolutional neural network structures and the construction of various face antispoofing databases, the deep learning‐based face antispoofing algorithm has become the main method in the FAS field. However, the generalization performance of the current multimodal face antispoofing algorithm is poor, and the recognition performance of the model on different datasets is quite different. Therefore, we design a multimodal face antispoofing framework based on a multifeature transformer (MFViT) and multirank fusion (MRF). First, we use a vision transformer structure, MFViT, for multimodal face antispoofing and a combination of modalities to capture the distinguishing characteristics in each modality. Second, we design a multidimensional multimodal fusion module, MRF, according to the various modal fusion characteristics obtained by the MFViT to fuse modal information in different dimensions more effectively. Evaluation results indicate that framework we designed achieves an average classification error rate (ACER) of 1.61% on the CASIA‐SURF dataset and an ACER of 6.5% on the CASIA‐SURF CeFA dataset.