Towards fusing gaze estimation and object prediction: What are you looking at?
Àlvaro Nieva-Suàrez, Marta Marrón-Romera, Cristina Losada-Gutiérrez, Irene Guardiola-Luna · Engineering Applications of Artificial Intelligence · 2025
Gaze Object Prediction (GOP) is a novel task that aims to evaluate which object is being looked at by humans. There are many applications for this task within the image understanding topic, but unfortunately, there are not many works that address it. In this paper, we propose a model named Gaze Estimation and Object Prediction (GEOP) that performs GOP, fusing the object detection and gaze following tasks. Specifically, we use object detection features to carry out both tasks, avoiding the extra computational cost added by using different branches. In addition, we include a dual gaze regression approach to improve the gaze following performance. A deeply analysis of the influence of auxiliary prediction heads to improve the detection accuracy has been carried out to optimize the proposed architecture. In this work, we also propose a novel metric called Average Precision Bounding Box Looked at (AP BL ) that explores further the GOP task. Extensive experiments on the Gaze On Object (GOO) dataset prove the performance of the proposed model in object detection, gaze following, and GOP tasks, demonstrating that the proposal outperforms other state-of-the-art approaches on both average error (Distance and Angle) and GOP metrics.