Motion-Based Convolutional Neural Networks for Super-Resolution from Compressed Videos
Han-Yuan Chang, Hsiang-Yu Hsieh, Yih-Chuan Lin · 2023
In this paper, an improved super-resolution method is proposed for compressed videos, which integrates the multi-image super-resolution (MISR) method into standard video decoders such as H.26x and AV1 to employ the reference frame (RF) and motion vector (MV) information from video decoder to gain better super-resolution efficiency and performance. The proposed motion vector-based multi-image super-resolution (MVSR) is trained by the convolutional neural networks (CNN) model, which consists of a series of convolution layers and is learned to identify the relationship of image features between the ground truth high-resolution (HR) and super-resolution (SR), and generates the SR frames that could improve the quality of the final output frame. The experimental results have shown that the proposed MVSR is capable of enhancing the SR quality over 1dB in terms of peak-to-noise ratio (PSNR) when compared to other similar approaches.