Reconstruction Algorithm for Lost Frame of Multiview Videos in Wireless Multimedia Sensor Network Based on Deep Learning Multilayer Perceptron Regression
Ting‐Lan Lin, Hua-Wei Tseng, Yangming Wen, Fu-Wei Lai, Ching-Hsuan Lin, Chuan-Jia Wang · IEEE Sensors Journal · 2018
Wireless multimedia sensor network (WMSN) is important for environmental monitoring. When the sensors are used as cameras, the network can be regarded as a multiview video system. The Packet loss may occur when the multiview videos are transmitted wirelessly. When the video frames are lost during transmission, a frame reconstruction method is needed in the decoder to estimate the missing pixels. In the proposed work, a reconstruction algorithm for lost frame of multiview videos in the WMSN based on deep learning methods is presented. A novel pixel estimation algorithm is developed using multilayer perceptron regression (MPR) with the deep learning method. Furthermore, a modified inpainting method is proposed with the use of the information from the optical flow algorithm with the neighboring available frames. Compared with the state-of-the-art method, the proposed MPR method with the traditional inpainting method increased the average peak signal-to-noise ratio up to 5.62 dB. The combination of the proposed MPR method with the proposed inpainting method outperformed previous proposed combination up to 8.32 dB on average, showing the significance of the proposed inpainting method.