A Hybrid Approach for Accurate Viewport Prediction in 360° Video Streaming
Junchao Shao, Z.C. Li, Jinpeng Song · 2025
Immersive panoramic video streaming plays a crucial role in VR/AR applications, but its high-resolution content places significant demands on network bandwidth and computational resources. This paper introduces a novel real-time viewport prediction model based on a dual-stream hybrid architecture designed to address these challenges effectively. The model consists of two streams: a user behavior stream and a video content stream.In the user behavior stream, an extended Long Short-Term Memory network (xLSTM) captures long-term dependencies in user head movement trajectories, overcoming the gradient vanishing problem typical of conventional LSTM models. The video content stream employs a Spherical Convolutional Neural Network (SphereCNN) to extract robust spatial features from equirectangular-projected panoramic video frames, effectively compensating for geometric distortions, particularly in polar regions.A dynamic fusion module then combines the temporal and spatial features to provide accurate, low-latency viewport predictions. Experimental results demonstrate that the proposed model significantly outperforms traditional methods in prediction accuracy, processing efficiency, and real-time performance, thereby improving the overall user experience in panoramic video streaming systems.