Face Shows Your Intention: Visual Search Based on Full-face Gaze Estimation with Channel-spatial Attention
Song Liu, Xiangdong Zhou, Xingyu Jiang, Haiyang Wu, Yu Shi · 2021
Visual search is the process that humans use visual perception to recognize targets of interest among multitudinous objects, which is a challenging research topic in computer vision.In contrast to previous works that take the overt gaze signal as input to predict the target of visual search with computational models, we proposed a visual search network based on full-face gaze estimation with channel-spatial mechanism, which can directly predict the user's search objects from the full-face images without extra obtaining the prohibitive intermediate gaze data.We seamlessly integrate the gaze information generated by the full-face gaze estimation module and the semantic information of the scene image into the visual search network that can directly infer the user's search intention.We demonstrate the effectiveness of our method for visual search task in real-world settings, and illustrate that directions for future research on full-face based human visual cognition.