Video Compression with Arbitrary Rescaling Network
Mengxi Guo, Shijie Zhao, Hao Jiang, Junlin Li, Li Zhang · 2023
We propose a practical downsampling compression scheme, as shown in Fig 1, to use a neural network as a pre-processing module for traditional codecs to improve compression performance. Specifically, we propose a neural network-based pre-processing method to improve video compression performance by downsampling high-resolution videos. Our method, called the rate-guided arbitrary rescaling network (RARN), uses a pre-trained entropy module [1] to estimate bitrate and guide the sampling process. We also use a transformer-based virtual codec (TVC) to simulate the performance of traditional codecs, using a swing-transformer-based invertible neural network to learn distortion from standard codecs, and the cyclic shift attention [2] in the network can approximate the prediction modes of HEVC. Our method, which is compatible with standard codecs (HEVC and VVC), performs well in various sampling ratios according to experimental results.