M-DVC: Moderate Deep Video Compression Embracing the Metaverse New Era
Zhenxing Li, Qiong Liu · 2023
In the Metaverse new era, video compression methods are required to provide low latency, low complexity, and high coding performance. In the low-latency scenarios, utilizing multiple reference frames in learned video compression methods has shown their capabilities of improving coding performance. However, these sophisticated methods generally yield to high complexity, mismatching the real-world Metaverse applications. In this paper, we propose a moderate deep video compression (M-DVC) method to balance the coding performance and complexity. Specifically, we first calculate the motion vector (MV) between the current frame and the previous frame to capture the motion information. Then, we introduce a multi-reference frame motion compensation (MFMC) module to exploit the high correlation among multiple frames and obtain more accurate prediction frames, thereby enhancing coding performance. In our M-DVC, we strive to utilize lightweight network structures to achieve a balance between coding performance improvement and reduced time complexity. Experimental results demonstrate that our M-DVC achieves a favorable trade-off between time complexity and coding performance. Specifically, on the HEVC Class E dataset, our M-DVC achieves a bitrate saving of 43.8% and 55.2% compared to H.264 when evaluated based on PSNR and MS-SSIM, respectively. In contrast, DVC achieves a bitrate saving of only 34.9% and 28.5% compared to H.264 based on PSNR and MS-SSIM evaluations, respectively. Additionally, our M-DVC introduces a minimal increase of only 5.9% in terms of time complexity compared to DVC.