Neuromorphic Event-based Face Identity Recognition
Goncalo Moreira, André Rui Graça, Bruno Silva, Pedro Martins, Jorge P. Batista · 2022 26th International Conference on Pattern Recognition (ICPR) · 2022
Faces in motion contain rich information about a myriad of social cues, including identity. However, is still unknown how much information about a person’s identity is conveyed by different kinds of facial movements and the precise role of facial motion as a cue for identity is still under debate. Open questions include how much information about identity is embedded in facial movements, and how efficiently a perceptual system extracts those cues toward identity. We address these questions by exploring the potential of Neuromorphic Vision Sensors to perform identity recognition through facial dynamics derived from speech and simultaneously validate how contributory facial motion is for human face identity categorization. As far as we know, this is the first work that evaluates the aptitude of the technology offered by neuromorphic cameras for completing the task of facial identity recognition. A one-stream inflated 3D convnet was used to evaluate the ability of different frame-based events encoding strategies in classifying identity through Short-Time Facial Motion Tokens (ST-FAMOTO). A consistent set of evaluations was conducted using two small datasets of individuals engaged in speech tasks, specifically text reading and speech-driven visual passcode. Results reported consistently consolidate the contribution of facial dynamics in the process of identity recognition and the usefulness of neuromorphic sensor devices in acquiring identity discriminatory facial micro-movements. Additionally, it is also proposed the first Neuromorphic Vision Dataset for Speech Induced Facial Dynamics (NVSFD Dataset), created in the scope of this work and available to facilitate future third-party investigation on the topic.