LiveAE: Attention-based and Edge-assisted Viewport Prediction for Live 360° Video Streaming
Zipeng Pan, Yuan Zhang, Tao Lin, Jinyao Yan · 2023
Viewport prediction plays a crucial role in live 360° video streaming as it determines which tiles should be prefetched in high quality, thereby significantly impacting the user experience. However, the current approach to viewport prediction, which integrates content-level visual features with the viewer's head movement trajectory, faces the challenge of striking a balance between prediction accuracy and computational complexity. In this paper, we propose LiveAE, a novel attention-based and edge-assisted viewport prediction framework for live 360° video streaming. Specifically, we employ a pre-trained video encoder called Vision Transformer (ViT) for general visual feature extraction and a cross-attention mechanism for user-specific interest tracking. To address the computational complexity issue, we offload the aforementioned content-level operations to an edge server while retaining trajectory-related functions on the client side. Extensive experiments show that our proposed method not only outperforms state-of-the-art algorithms but also ensures the real-time requirements of live 360° video streaming.