Predicting User Attention in Virtual Reality: Integrating Deep Learning with Real-Time Gaze Forecasting
Esra Demir, Yusuf Çelik, Jian Chi · 2024
The field of Gaze Forecasting in Virtual Reality (VR) is rapidly evolving, leveraging advances in Human-Computer Interaction (HCI) to create more immersive and responsive virtual environments. A critical research question that remains insufficiently addressed is whether real-time gaze forecasting can be improved through the integration of deep learning techniques. This paper aims to bridge that gap by proposing a novel framework that combines advanced deep learning models with real-time eye-tracking data to predict user attention in VR environments. The proposed method employs a convolutional neural network (CNN) augmented with a recurrent neural network (RNN) to capture both spatial and temporal dynamics of gaze behavior. Furthermore, the system is designed to operate in real-time, ensuring that gaze predictions can be promptly utilized for adaptive content delivery and user interface adjustments. Our contributions include the development of a new dataset specifically tailored for VR gaze prediction, the introduction of a hybrid deep learning architecture, and a real-time implementation capable of running on consumer-grade hardware. Experimental results demonstrate that our method significantly outperforms existing state-of-the-art approaches in both accuracy and responsiveness, achieving up to a 20% improvement in prediction accuracy. User studies confirm that the enhanced gaze forecasting translates to a more intuitive and engaging VR experience. This work not only advances the technical boundaries of gaze prediction but also opens new avenues for more personalized and adaptive VR applications.