Non-Local Spatiaotemporal Two-Stream Network for Video Action Recognition
Jie Zheng, Yu Michael Zhu · 2019
Current video action recognition methods mainly used repeated convolution and recurrent operation to obtain the spatial and temporal information. But these operations only process local neighborhood in space and time and are not sensitive to global information, which may lead to wrong prediction for some large-space and long-time span actions. So in this paper, we proposed a non-local 2D spatiotemporal two-stream network, and incorporate the non-local module to I3D spatiotemporal two-stream network. This module calculates the embedded Gaussian response at a position as a weighted sum of the features at all positions in the input feature maps. The obtained non-local information enhances the global feature in both space and time effectively. In the proposed network, we first use several low-level layers to obtain low-level feature maps, then passes the non-local module to obtain non-local information. Finally, using the subsequent layers to get prediction result. The performances of the proposed 2D model and the improved I3D model were evaluated on UCF-101 dataset and HMDB-51 dataset, and achieve 95.27% and 69.61% recognition accuracy respectively.