Gaze-Adaptive Subtitles for 360° Videos in Virtual Reality
Tianrui Hu, Wenjun Hou · 2024
Subtitles (captions displayed on the screen) play a crucial role in cross-lingual multimedia content, serving as valuable aids for individuals who are deaf or hearing-impaired, as well as in situations involving unknown languages or loud environments. Subtitles are equally indispensable when viewing 360° immersive videos in virtual reality (VR). However, the experience of watching 360° videos in VR differs significantly from that on desktop screens. Hence, we propose a method for presenting VR 360° video subtitles that leverages users’ gaze data as indirect input to dynamically adjust subtitle positions in real-time. Our research investigates the impact of adaptive subtitles compared to traditional subtitles on users’ experiences when viewing 360° videos in VR environments.Research findings suggest that the proposed adaptive subtitle placement method is more effective than traditional subtitles located below the field of view in reducing visual fatigue and enhancing the watching experience for users. Furthermore, in comparison to conversational 360° videos featuring speaking subjects, adaptive subtitles offer greater optimization and improvement in scene-oriented videos with voice-overs.