Recognizing facial expressions of emotion using action unit specific decision thresholds
Mustafa Sert, Nükhet Aksoy · 2016
Automatic analysis of facial expressions of emotion has been an active research topic of computer vision and machine learning communities. Building person and culture independent models is the main challenge for both communities. We need effective, yet adaptive methods to cope with this challenge. In this study, we present a novel method for recognizing facial expressions of emotion based on the developed facial action unit (AU) detector and using rule-based reasoning. Our AU detector, also referred to as Adaptive Decision Thresholding based AU detector (ADT-AU) performs decision threshold analysis for each AU using a fitness function to learn the optimum decision threshold of the binary learning method. We choose the Support Vector Machine (SVM) algorithm as the binary method and utilize Active Appearance Model (AAM) features. Using ADT-AU detector we detect 17 AUs occurring alone or in combination and recognize six facial expressions of emotion (e.g., surprise, fear, happiness, etc.) using the prototypic and major variants of AUs by our rule-based emotion classifier. Our experiments on Extended Cohn-Kanade (CK+) dataset show that the proposed method outperforms baseline method that uses standard decision threshold and provides significant improvements on most of the facial expressions with an average F1-score of 5.59%.