Semantic Video Segmentation with Dense-and-Dual Warping Spatial Features
Mehwish Awan, Jitae Shin · 2020
We propose a fast and accurate network for video semantic segmentation based on dual-warping of keyframe spatial features for capturing both sparse and dense motions between frames. The proposed dual-warping operation precisely propagates the keyframe features for subsequent frames, particularly improves the propagations of fast-paced spatial contents among frames. The spatial features from deep convolutional neural network are first warped based on the optical flow between current frame and previous frame. Then, we compute warped flow field between all frames from keyframe to current frame. The previously warped features are next warped with computed warped flow field to rectify the fast-moving features missed in initial warping operation. Our network is trained in end-to-end manner, and the accuracy is significantly increased by enhancing the propagation of keyframe features with high throughput. We evaluated our approach on Cityscapes benchmark video dataset.