FACSvatar
Stef van der Struijk, Hung‐Hsuan Huang, Maryam Sadat Mirzaei, Toyoaki Nishida · 2018
Embodied Conversational Agents often employ advanced multimodal analysis of human users for affective inference, however, its facial expressions in response are often pre-made or coded animations. Generated data-driven facial animations have the advantage that they can be more natural and do not require a database at run-time. The open source modular framework FACSvatar is presented, which processes and animates FACS based data in real-time. Tools that create 3D human models are supported and facial data can be visualized in popular tools in the gaming industry. All functionality is split-up into modules set-up in a publisher-subscriber pattern to provide easy integration with other platforms. A deep learning module for real-time generation of AU data for data-driven animation is implemented. A user evaluation of their expressions being animated through our framework was done. Their ratings were slightly positive, but more improvements have to be made in terms of data quality and individual fine-tuning. Also, the modules' latency and performance have been measured. On average, FACS data is visualized in 28.55 ms.