Multi-modal/multi-scale convolutional neural network based in-loop filter design for next generation video codec
Jihong Kang, Sungjei Kim, Kyoung Mu Lee · 2017
In this paper, we propose a novel in-loop filter design for video compression. Our approach aims to replace existing deblocking filter and SAO (Sample Adaptive Offset) of HEVC standard with multi-modal/multi-scale convolutional neural network (MMS-net). The proposed CNN architecture consists of two sub-networks of different scales. An input image is down-sampled first and restored through the lower scale network, then the output image from it is fed into higher scale network concatenated with the original input image. Moreover, to boost the restoration performance, the proposed architecture utilizes information resides in the coded sequence. Specifically, the compression parameters from coding tree units (CTU) are exploited as input to CNN, which helps to alleviate blocking artifacts on the reconstructed images. In the experiments, our method reduces the average BD-rate by 4.55% and 8.5%, respectively, compared with the conventional neural network based approach [1] and HEVC reference software HM16.7 [2] in `All Intra - Main' configuration.