Emotion Recognition Based on Multi-View Body Gestures
Zhijuan Shen, Jun Sheng Cheng, Xiping Hu, Qian Dong · 2019
Body gesture, a crucial component of "body language", remains less explored to recognize emotion while face expression-based and speech-based approaches are widely investigated. In this paper, we introduce an exploratory experiment to recognize emotion using deep learning only from body gestures. 43,200 multi-view RGB videos of simplified body gestures and their neutral control groups are captured from 80 humans using Hikvision network cameras to support the experiment. A novel approach is proposed to use deep neural network fuse skeleton and RGB features only using single-modality RGB video data. Experimental results show our approach achieves substantial improvements both in individual categories and overall and is provided with stronger generalization capability as well.