Towards Efficient Video Inpainting Based on Implicit Neural Representation
Shuxuan Fu, Yuxin Xi, Fangming Jing, Jing Wang · 2024
Video inpainting is an important method to improve video quality and viewing experience. Traditional video inpainting methods require manual design, which has high operation costs and low efficiency. The deep learning method based on implicit neural representation provides an efficient automatic solution for video inpainting. It can improve the accuracy and effect of video inpainting by automatically learning the characteristics of the video and inpainting the video according to these characteristics. However, the scale of the embedding (feature) of the encoded video by the encoder is often small (16×2×4). As the input of the decoder, the embedding can only provide a small amount of feature information for the decoder, which greatly restricts the ability of video neural representation and further affects the effect of video inpainting. In this work, we construct a novel method of implicit neural representation for video inpainting, named I-NeRV, which enables a large-scale embedding (16×8×16) to enrich the information of the embedding. Meanwhile, the random mask mechanism is integrated into the coding part of neural representation, further improving the network feature extraction ability. Comprehensive experiments have proved that the video neural network in this paper has achieved significant optimization results in video inpainting tasks. On the representative dataset, video inpainting is more effective than SOTA, with a 3.47 PSNR improvement in inpainting metrics.