FPX-NVC: An FPGA-Accelerated P-frame Based Neural Video Coding System
Chuanmin Jia, Xinyu Hang, Wenbo Liu, Shanshe Wang, Siwei Ma · 2022 IEEE International Conference on Visual Communications and Image Processing (VCIP) · 2022
Hardware-oriented neural video coding (NVC) has been an emerging research topic recently. In this work, we present a P-frame based NVC system by leveraging the parallel neural inference capability of resource-constrained embed platforms, such as field programmable gate array (FPGA). The proposed system, namely FPX-NVC, consists of multiple resolution video capture, reconfigurable and integerized neural video codec and display. Our neural codec enjoys the learned unidirectional inter-predictive coding by jointly optimizing the implicit motion representation (IMR), feature domain motion compensation, and the corresponding prediction error compression, realizing end-to-end on-board encoding and decoding. Our system supports end-to-end video data operation flow from capturing to neural coding and display. To our knowledge, the FPX-NVC is the first inter-predictive coding (P-frame) based hardware NVC system.