Biologically inspired deep neural network models for visual emotion processing
Peng Liu, Ke Bo, Yu-Jung Chen, Andreas Keil, Mingzhou Ding, Ruogu Fang · Network Neuroscience · 2026
Abstract The perception of opportunities and threats in complex visual scenes represents one of the main functions of the human visual system. The underlying neurophysiology is often studied by having observers view pictures varying in affective content. While deep neural networks (DNNs) have shown promise in modeling visual recognition of objects, their capacity to model visual affective processing remains to be better understood. In this study, we proposed a biologically inspired deep neural network model, referred to as the visual cortex amygdala (VCA) model, for this purpose. The model integrates a vision transformer module for visual encoding and an amygdala-mimetic module that incorporates an anatomical hierarchy and self-attention-based computational mechanisms for affective decoding. We evaluated the model along three dimensions: (a) predictive accuracy for emotional valence and arousal, (b) representational alignment with human amygdala activity, and (c) internal organization of emotion representation within the model. The results showed that (a) the model can predict with high accuracy human emotion ratings on 1,182 images from the International Affective Picture System (IAPS) dataset (valence: r ≈ 0.9; arousal: r ≈ 0.7), (b) the model’s internal representations aligned with functional magnetic resonance imaging (fMRI) data from the human amygdala, and (c) at the single model neuron level, the amygdala module evolved emotion selectivity; in addition, at the model neural population level, deeper layers of the amygdala module developed representational geometry that is progressively more aligned with affective dimensions. We also explored the effect of visual encoding and the effect of structure and computational mechanisms on emotional assessment.