On the robustness of audiovisual liveness detection to visual speech animation
Jukka Komulainen, Iryna Anina, Jukka Holappa, Elhocine Boutellaa, Abdenour Hadid · 2016
Audiovisual speech synchrony detection is an important liveness check for talking face verification systems to make sure that the (pre-defined) content and timing of the given audible and visual speech samples match. Nowadays, there exists virtually no technical limitations for combining transferable facial animation and voice conversion (or synthesis) to create an ultimate audiovisual artifact that is able to spoof even advanced random challenge-response based liveness detection. In this study, we investigate the performance of the state-of-the-art text-independent lip-sync detection techniques under presentation attacks consisting of audio recordings of the targeted person and corresponding animated visual speech. Our experimental analysis with three different photo-realistic visual speech animation techniques reveals that generic synchrony models can be fooled even with underarticulated but synchronized lip movements. Thus, measuring audio-video synchrony or content alone is not enough for securing audiovisual biometric systems. Our preliminary findings suggest though that adaptation of person-specific audiovisual speech dynamics is one possible approach to tackle these kinds of high-effort attacks.