An Investigation of Latency-Accuracy Trade-off in Inter-frame Video Prediction using Quantized CNNs
T. De Silva, Pradeepa Yahampath · 2023
Real-time video streaming has become the largest portion of internet traffic in recent years. Therefore, improving the efficiency of video coding remains an important research issue. Modern video codecs perform inter-frame prediction by motion estimation. However, inter-frame prediction is one of the most computationally expensive and time-consuming operations in video coding. Convolutional neural networks (CNN) have been used in recent research for inter-frame prediction tasks. The CNN architectures in previous work use floating point arithmetic whereas motion estimation in video codecs only use integer arithmetic. Thus, inter-frame prediction using CNNs instead of motion estimation may not always result in better time complexity. Floating point CNNs can be quantized into integer CNNs. Integer CNNs can reduce network latency but can also result in a loss of prediction accuracy. In this paper, we investigate the latency vs accuracy trade-off of quantized CNNs in inter-frame bi-prediction. We present experimental results which demonstrate that the integer CNN is at least 5% faster than the floating point CNN, while the prediction quality degradation of the integer CNN is no more than 0.6 dB in PSNR.