Interpersonal Relationship Detection Using Multi-Head Graph Attention Networks With Multi-Feature Fusion
Simge Akay, Duygu Çakır, Nafiz Arıca · IEEE Access · 2025
The ability to automatically detect and understand interpersonal relationships from visual data represents a fundamental challenge in computer vision and social signal processing. While significant advances have been made in face recognition and object detection, understanding the complex dynamics of human relationships requires a detailed approach that can integrate multiple visual and contextual cues. This paper presents a novel Multi-Head Graph Attention Network (MHF-GAT) with Multi-Feature Fusion for interpersonal relationship detection (IRD) from images. The proposed architecture decomposes heterogeneous visual and semantic features into three specialized modules: local features capturing direct interpersonal cues (e.g., age differences, expression similarity), contextual features modeling environmental information (e.g., scene, objects), and semantic features derived from image captions. By representing features as graph nodes and employing multi-head attention mechanisms within each module, MHF-GAT effectively captures complex feature interactions while reducing redundancy and overfitting risks. The model’s performance is evaluated on two benchmark datasets: the Interpersonal Relation Dataset (IPR) and People in Social Context (PISC). Experimental results demonstrate state-of-the-art performance, achieving 87.18% accuracy on IPR and 77.1% mean Average Precision (mAP) on PISC. Ablation studies and attention visualization reveal that the modular architecture significantly improves feature discrimination and relationship classification compared to single-graph approaches.