Learning Prediction of Emotional Change on Behaviors
Jiahao Yu, Wei‐Shi Zheng · 2019
We are interested in the topic about the prediction of emotional change caused by other people's behaviors. Although information of other people's behaviors is intuitive and important for arousing emotional change, existing works on prediction of emotional change rarely take it into consideration. In this work, we propose a multimodal deep network, called Emotional Change Prediction Network (ECPNet), for predicting emotional changes caused by behaviors of other people. ECPNet can learn useful cues from visual and acoustic information and extract both local and global spatiotemporal multimodal features of behaviors. Furthermore, the proposed network can learn the temporal relations of the behaviors in videos which help us to explore the behaviors' cumulative effects on other people's emotions. In order to quantify our model, we establish the first dataset on emotional change prediction by behaviors. Experiment results show that our approach can effectively model emotional change and perform better than other baselines.