The Influence of Text-guidance on Visual Attention
Yinan Sun, Xiongkuo Min, Huiyu Duan, Guangtao Zhai · 2023
Visual attention analysis and prediction have long been important tasks in computer vision and image processing. However, images often come along with various text descriptions in real applications, while the influence of these text-guidances on the visual saliency of corresponding images have rarely been studied. Therefore, in this paper, we mainly focus on the problem of whether and how the text-guidance influences the visual attention during image viewing, and perform subjective experiments, qualitative and quantitative comparisons as well as model evaluations on this new task. Specifically, we first conduct eye tracking experiments on 300 images under text-visual (TV) and visual (V) test conditions, respectively. Based on the subjective experiments, we perform qualitative and quantitative comparisons between the visual attention data collected under TV and V conditions, and conclude that the text-guidance can significantly influence the visual attention, especially when the text-described target is a non-salient object. Finally, we evaluate the existing saliency models on our database, and find that existing models cannot well handle this text-induced saliency prediction task. Our constructed database will be publicly available to facilitate future research.