Identification of Individuals Using Multimodal Data and LSTM Neural Networks
João Victor Campos de Negreiros, Caíque V. L. Muniz, David Laureano Dos Santos, Fabio R R Santos, M. G. F. Costa, Cícero Ferreira Fernandes Costa Filho · 2023
Currently, biometric systems are widely used in the process of identifying individuals. These systems can be classified as unimodal or multimodal, depending on the number of biometric characteristics or modalities used. Multimodal systems tend to provide authentication with higher hit rates and greater robustness against fraud. This paper proposes a multimodal biometric identification system based on face and voice features using long-short-term-memory recurrent networks and a fusion method at feature level. The face features were extracted using an autoencoder neural network, while for voice, the features are the Mel Frequency Cepstral Coefficients. Data from 50 individuals (37 males and 13 females), taken from the MOBIO database, were used. The best results obtained for each system were: accuracy of the unimodal face system = 97.79%; accuracy of the unimodal voice system = 79.98%; accuracy of the multimodal face and voice system = 99.47%.