Video frame interpolation via residual blocks and feature pyramid networks
Xiaohui Yang, Haoran Zhang, Zhe Qu, Zhiquan Feng, Jinglan Tian · IET Image Processing · 2022
Abstract Various deep learning‐based video frame interpolation methods have been proposed in the past few years, but how to generate high quality interpolated frames in videos with large motions, complex backgrounds and rich textures is still a challenging issue. To deal with this limitation, a frame interpolation method based on residual blocks and feature pyramids is proposed. U‐Net is the main architecture of our method, which can capture multi‐layer information, segment objects from the background and obtain parameters with motion information to guide frame interpolation. However, the upsampling and subsampled of U‐Net will lose important information. In order to acquire more detailed contextual information, shortcut connection is used in the encoder basic module. At the same time, feature pyramid network is employed to capture features at different scales of the decoder to improve the representation of inter‐frame spatial‐temporal features. The experimental results show that the proposed method outperform the baseline methods in both of objective and subjective evaluations on different datasets. In particular, the method has obvious advantages on datasets which contain complex background.