HPGER: Integrating Human Perception Into Group Emotion Recognition
Abolfazl Farahani, Navid Hashemi Tonekaboni, Khaled Rasheed, Hamid Reza Arabnia · 2022
Over the past few years, deep neural networks have been widely employed for representation learning and achieved remarkable success in many computer vision tasks, such as visual sentiment analysis and emotion recognition. However, identifying image sentiments similar to what humans do is challenging due to the complexity of raw images and the intangible nature of human visual perception. Besides, training a deep learning model from scratch for a complex task such as group emotion recognition is very time-consuming and requires a large amount of labeled data representing the total population which is practically infeasible. In addition, creating annotated data for each specific task is costly and sometimes impossible. So instead, we can use the knowledge extracted by a model trained on related existing labeled datasets. To address the above challenges, we propose an end-to-end group emotion recognition framework that integrates human perception learned from human eye fixation data to efficiently and effectively extract the sentiment of input images and classify them as positive, negative, or neutral. The proposed architecture aims to leverage the information in eye fixation data to learn how humans perceive and interpret the visual world in a free-viewing task. Our framework can be utilized in any free-viewing task, potentially reducing the need for a large amount of labeled data and speeding up the training phase. In the following sections, The following sections outline the initial steps and describe different parts of our architecture.