Research on Face-Voiceprint Correspondence Learning with Aux-Output Supervised Convolution Neural Network
Cong Yanping, Cao Linlin · Pattern Recognition and Image Analysis · 2022
Abstract According to the problem that the security of face-voiceprint login system is not high, using a face photo or video containing a real face and a snippet of fake speech can sometime cheat the system because of the faiblesse of voiceprint recognition. Therefore, we have proposed a method of using the correspondence of individual’s face and its voiceprint to improve the security of the face-voiceprint login system, which can efficiently prevent the cheat of using one’s face photo and another’s recorded speech. We add the face-voiceprint correspondence check after the face recognition and voiceprint recognition in the system. The model is consisted of several blocks of CNN for extracting the face feature and voiceprint feature, a fusion layer for concatenating the two modalities, and a layer of attention for searching the correspondence of face feature and voiceprint feature. To make the model learn better the task, we add some aux-output at the end of model for multi-task learning, which makes the model learn easily the relations hidden in the face and voiceprint. Experiments show that the method can extract both face and voiceprint features and learn the correspondence of face and voiceprint of a person to improve and perfect the security of a face-voiceprint login system.