Gaze estimation based on machine learning
Andoni Larumbe-Bergera · 2024
This thesis, developed within the framework of the GI4E group, focuses on the development of a gaze estimation algorithm for off-the-shelf video-oculography (VOG) systems. The limitations of existing eye tracking technology and gaze estimation methods in off-the-shelf systems are highlighted. The main contribution of this thesis is the development of a gaze estimation algorithm divided into two main blocks: the first block for facial landmark detection and the second block for estimating the Point of Gaze (PoG) from a feature vector generated using these landmarks. Due to significant advances in the field of machine learning, it has been decided to employ this type of techniques for both blocks. A review of state-of-the-art methods using machine learning and deep learning for facial landmark detection is conducted, along with an exploration and summary of state-of-the-art algorithms applied to gaze estimation. For the facial landmark detection block, two models are implemented, a first one based on cascade regression methods and another based on neural networks. Both models are compared on various databases, analyzing the strengths and weaknesses of each. Additionally, a comparison between the proposed method and the state of the art is conducted, showcasing the superiority of our approach. Regarding the second block, a method for generating a feature vector that includes relevant information for gaze estimation is presented. Furthermore, several neural network-based models are proposed, and the use of synthetic data for training gaze estimation models is investigated. Finally, a method to adapt and calibrate the models trained with synthetic users to real data is proposed. The thesis concludes with a summary of its contributions and key findings. The integration of machine learning techniques, advanced algorithms, and synthetic data holds promising prospects for future research in this field.