Social-Guided Representation Learning for Images via Deep Heterogeneous Hypergraph Embedding
Yunfei Chu, Chunyan Feng, Caili Guo · 2018
Representation learning for images is widely recognized as critical to the performance of end tasks such as image classification and cross-modal retrieval. However, most existing methods extract features only from visual content, far from adequate in interpreting semantics latent in images. For social images, there also exists rich social context information, e.g. owners, tags and groups, which provides cues for interpreting semantics. In this paper, we propose a representation learning framework via deep heterogeneous hypergraph embedding (DHHE), considering both visual content and social contexts. In particular, images and their contexts are first represented as a heterogeneous hypergraph, which is then embedded into a low-dimensional space. To incorporate visual information and generalize for unseen images, we learn the mapping from visual content to the semantic space. We conduct experiments with the tasks of classification, cross-modal retrieval and recommendation, which demonstrates the effectiveness of our approach and the merits of social guidance.