Bi-directional prediction for end-to-end optimized video compression
Fabien Racapé, Jean Bégaint, Simon Feltman, Akshay Pushparaja · 2021
This paper presents and studies an end-to-end Artificial Neural Network (ANN)-based compression framework leveraging bi-directional prediction. Like traditional hybrid codecs in Random Access configuration, this codec processes video sequences divided in Groups Of Pictures (GOPs) in which each frame can be encoded in Intra or Inter mode. Inter frames, can be bi-predicted, i.e. using past and future previously decoded frames, the selection of the reference frame for prediction are signaled within the bitstream, allowing for efficient hierarchical GOP temporal networks. In particular, we study the benefits of optimizing the compression of the motion information prediction residuals using dedicated auto-encoder models in which the layers are conditioned based on the GOP structure. The network is trained fully end-to-end from scratch. The increase of compression efficiency shows the promises of implementing conditional convolution for bi-directional inter coding.