Probabilistic prediction of student affect from hand gestures
Abdul Rehman Abbasi, Matthew N. Dailey, Nitin Afzulpurkar, Takeaki Uno · 2008
Abstract — Affective information is vital for effective human-tohuman communication. Likewise, human-to-computer communication could be potentiated by an “affective barometer ” able to infer human affect using a machine vision system. For instance, during a classroom lecture, an affective barometer might provide useful feedback that a real or virtual instructor could use to improve pedagogical strategies. In this paper, we explore the feasibility of using students ’ unintentional hand gestures during a classroom lecture to predict their affective state. We propose a maximum a posteriori classifier based on a simple Bayesian network model. We then evaluate the classifier’s ability to predict one of four affective states from five hand gestures observed in video recordings of a classroom lecture. Using four-fold cross validation, we find that the model’s generalization accuracy is 100 % over cases where the student reported an affective state, and 79.4 % when we include cases where the student reported no affective state. The experiment demonstrates that there is a strong relationship between human affect and visually observable gestures. Future work will explore the applicability of these results in practical applications. Index Terms — Behavior recognition, Intelligent tutoring systems, Human-computer interaction, Probabilistic affect prediction,