Spherical Convolution-based Saliency Detection for FoV Prediction in 360-degree Video Streaming
P. Shuai, Jialu Hu, Zitong Li, Han Xiao, Shujie Yang, Changqiao Xu · 2023
Field of view (FoV) prediction is a crucial issue in 360° video streaming, which is the basis for selectively transmitting panoramic videos to reduce bandwidth. The saliency feature is a very important part of FoV prediction. The saliency area identifies a user’s region of interest (RoI) and reflects the user’s viewing behavior preference. The regular convolutional neural network (CNN) cannot effectively extract the spatial representation of panoramic video content because significant geometric distortion will be introduced after panoramic video projection, especially in polar regions. In this paper, we propose a depth neural network model based on spherical convolution, which can learn the spatial features of the 360° videos by encoding the distortion invariance into the architecture of CNNs. A series of experiments on the public 360° video saliency dataset show the proposed model outperforms the existing saliency models. Finally, we embed the proposed saliency network into a popular FoV prediction framework and propose a complete FoV prediction framework for 360° video streaming.