Gaze Estimation via Synthetic Event-Driven Neural Networks
Himanshu Kumar, Naval Kishore Mehta, Sumeet Saurav, Sanjay Kumar Singh · 2024
Event cameras, which capture pixel-level intensity changes asynchronously, offer minimal motion blur, high temporal resolution, and low power usage compared to traditional cameras. However, with limited availability and the high cost of real event data, synthetic event data has become a crucial alternative for applications such as retrospective gaze analysis in medical research, user behavior studies, and neurocognitive research. It allows simulation of experimental conditions, optimization of gaze estimation algorithms, and detailed scenario testing, including rare events, without costly real-world data collection. In this paper, we demonstrate that synthetic event data can match the performance of real event data in saccadic gaze prediction. To support this, we introduce a synthetic event-based gaze dataset focused on saccadic eye movements and propose a novel Dual ResUNet architecture designed to process consecutive frames with synthetic encoded events for gaze estimation. Our network achieves feature-level domain adaptation and temporally consecutive centroid prediction, delivering 99.20% accuracy within a 100-pixel radius. Our research underscores the potential of synthetic temporally encoded images for precise gaze vector prediction while effectively minimizing the domain gap with real event data.