CvTSRR: A Convolutional Vision Transformer Based Method for Social Relation Recognition
Shahana Shultana, Lin Li, Na Li · 2023
Social relations refer to the relationships or interactions existing between people, such as family members, friends, coworkers, and so on. Social relationship analysis can provide understanding of human behaviors and therefore lead to many applications. With more and more people sharing images online, effectively recognizing social relations from the visual data becomes critical. Although a good number of studies had been carried out for social relationship recognition, the results are far from perfection. Better models need to be designed for feature extraction and image comprehension. In this paper, we propose a new learning model—CvTSRR for recognizing social relationships in images. This model employs a convolutional vision transformer (CvT) architecture to extract visual features and a multi-head attention mechanism to identify social relationships. The model was evaluated on a benchmark dataset called PISC (People in Social Context), and its performance was compared with those from previous works. Experimental results show that CvTSRR is competitive with the state-of-the-art models and outperforms them in several aspects.