Light-Weighted Temporal Evolution Inference for Generative Face Video Compression
Zihan Zhang, Bolin Chen, Shanzhi Yin, Shiqi Wang, Yan Ye · 2024
Recently, Generative Face Video Compression (GFVC) has advanced the concept of Model-based Coding (MBC) with promising rate-distortion performance relying on the strong inference capabilities of deep generative models. In particular, GFVC can capture temporal evolution of face video using compact representations (i.e., 2D/3D key-points, facial semantics, compact feature), thus achieving the quality and bandwidth trade-offs for ultra-low bit-rate communication. However, there remains an unaddressed challenge, i.e., the existing GFVC models are not light-weighted and low-latency enough for practical applications. To address these obstacles, this paper proposes a practical lightweight scheme based on the Compact Feature Temporal Evolution (CFTE) model, which aims to provide insights into practical deployments and efficient inference. Specifically, the lightweight network architecture is built with depth-wise convolutions and Inverted Residual Blocks to lower the computational complexity. Moreover, a feature-level knowledge distillation is further introduced to improve the performance of lightweight student CFTE model. Experimental results demonstrate that our proposed lightweight GFVC model can achieve an obvious complexity reduction, whilst maintaining competitive rate-distortion performance.