A Suitable and Efficient Super-Resolution Network for Neural Video Delivery
Wenbin Tian, Qingmiao Jiang, Haolin Li, Zhiyi Chen, Jinyao Yan · 2024
Recently, Deep Neural Networks (DNNs) based methods for modern video delivery systems have been acquiring remarkable performance due to their lower bandwidth requirements and better video quality. Specifically, these methods divide a low-resolution video into chunks, train a specific super-resolution model for each chunk on the server, and then stream low-resolution video chunks and corresponding content-aware models to the clients. The client uses their computing power to run the inference of models to super-resolve the low-resolution video chunks. However, the heavy model can achieve better super-resolution quality due to its high overfitting ability, which substantially poses a challenge for client-side computation power, increasing storage and consuming more bandwidth resources for data transmission. On the other hand, although some existing lightweight networks can deploy well to resource-constrained devices, the quality of the video generated is significantly subpar. To reconcile this, we propose a Partial Convolution based padding-based Residual Feature Network (PCRFN). The main idea is to use three partial convolution based padding layers for residual local feature learning to simplify network architecture, which achieves a better trade-off between model performance and efficiency. Moreover, we also analyze the impact of different activation functions and attention mechanisms on our network, which is an excellent guideline for designing a lightweight network. Extensive experimental results show that the proposed PCRFN achieves a better trade-off against state-of-the-art methods in terms of inference times, visual quality, and model complexity. Besides, we also conducted experiments applying our network to the latest neural video delivery, achieving comparatively good video quality and faster inference times across different video lengths. Therefore, PCRFN is a more suitable and efficient super-resolution network for neural video delivery. The Code is available is at https://github.com/Wenbin-Tian/PCRFN.