A Study of Multimodal Identification Methods
Jianjian Feng, Jianguo Wang · 2023
The purpose of this paper is to investigate unimodal determination and decision layer fusion methods in multimodal face recognition and to evaluate the effectiveness of decision layer fusion. Experiments are conducted on the "VidTIMIT" and "MOBIO" datasets using convolutional neural networks and MFCC feature extraction techniques. The performance of three feature fusion methods, front-end fusion, intermediate fusion and decision layer fusion, is compared. The experimental results show that the recognition accuracies are 94.9%, 94.7% and 94%, 94.1% for the face and voice models only, respectively. However, the accuracy decreases slightly in the case of front-end fusion. The best accuracy was achieved when the decision layer fusion method was used, reaching 95.7% and 95.6%, respectively. This indicates that training face and voice print data separately and fusing classifier output scores can effectively improve face recognition accuracy. This study explores feature fusion methods in the field of multimodal face recognition and experimentally demonstrates the advantages of decision layer fusion in improving accuracy, pointing to a new direction for future research.