Optimal Tile-Based Encoding for 360-Degree Video Streaming
Wenxuan Gao, Chenglin Li, Haoran Lv, Wenrui Dai, Junni Zou, Hongkai Xiong, Xinlong Pan, Haipeng Wang · 2022
360-degree videos pose a challenge to their transmission due to the ultra-high resolution. To achieve efficient streaming, current tile-based schemes divide the 2D projection of spherical frames into small rectangular tiles, which are treated as individual sources for encoding and then delivered according to user's actual viewport. However, the trade-off between transmission and encoding efficiency, as controlled by encoding parameters such as tile sizes and quantization steps, has not been fully studied yet. In this paper, we propose a tile-based encoding scheme to strike this trade-off, by selecting the optimal encoding parameters that minimize the distortion of video displayed for user's viewport under a given transmission capacity. Specifically, we quantify the transmission efficiency by estimating the average number of pixels required for transmission to cover the user's viewport, while a CNN-based predictor is trained to predict the encoding efficiency based on video content characteristics, expressed as number of bits needed for encoding one pixel. We then formulate a rate-distortion optimization problem for tile-based encoding, which is solved efficiently by decreasing the feasible set of encoding parameters. Simulation results demonstrate that the proposed optimal tile-based encoding outperforms both fixed tiling and state-of-the-art adaptive tiling schemes in terms of rate-distortion behavior.