Content-Aware Multi-task Neural Networks for User Gender Inference Based on Social Media Images
Ryosuke Shigenaka, Yukihiro Tsuboshita, Noriji Kato · 2016
To estimate demographic attributes such as gender and age of social media users from images posted by the users is a challenging problem because the demographic attributes are directly not shown in images. For such problem, prior approaches can be roughly separated into two types: one approach uses concept detection to detect pre-defined visual concepts which are then used as meta-data to estimate demographic attributes and the other approach directly uses content features such as Fisher Vector [19] which are extracted from images. In this paper we consider the way of combining these two approaches. We propose Multi-task Bilinear Model for integrating the detected concepts with the content features. In our proposed method, both the concept detector and the feature extractor can be jointly learned with end-to-end fashion. We evaluated the proposed method for the task of estimating user gender from Twitter images and found that it outperformed other baseline methods.