Inferring truth from multiple annotators for social interaction analysis
Gokul Chittaranjan, Oya Aran, Daniel Gática-Pérez · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · 2011
This study focuses on incorporating knowledge from multiple annotators into a machine-learning framework for detecting psychological traits using multimodal data.We present a model that is designed to exploit the judgements of multiple annotators on a social trait labeling task.Our two-stage model first estimates a ground truth by modeling the annotators using both the annotations and annotators' self-reported confidences.In the second stage, we train a classifier using the estimated ground truth as labels.Our experiments on a dominance estimation task in a group interaction scenario on the DOME corpus, in addition to synthetically generated data, give satisfactory results, outperforming the commonly used majority voting as well as other approaches in the literature.