A Lightweight Dynamic Gesture Recognition Network with Spatio-Temporal Attention
Xingyu Li, Lei Yang, Yanhong Liu · 2023
As one of the simplest and most natural ways of human-computer interaction, gestures can effectively convey information through silent hand movements and facilitate our production and life in combination with intelligent devices. Accurate and effective gesture recognition technology is an effective way to improve the efficient work of intelligent human-computer interaction devices in complex environments. However, due to the influence of complex surroundings and image noise, accurate prediction of dynamic gestures still faces certain challenges. Recently, benefiting from the end-to-end feature extraction capabilities of convolutional neural networks (CNN), deep learning has achieved good results in different dynamic gesture recognition tasks. However, it still has certain shortcomings in learning spatio-temporal semantic features to identify effective regional features. In order to accurately recognize continuous dynamic gestures in video, a dynamic gesture recognition network based on spatio-temporal attention mechanism is proposed to realize end-to-end dynamic gesture recognition. Aimed at the problem of information loss in the process of spatial and temporal feature extraction of video frames, a 3D residual convolution neural network is used to simultaneously model the temporal relationship and spatial information between frames. In order to obtain discriminative feature information, a spatio-temporal attention module (STA) is proposed to improve the ability of the network to extract temporal and spatial features. A large number of experiments on two public datasets show that the proposed dynamic gesture recognition network based on spatio-temporal attention can obtain the best recognition performance compared with other state-of-the-art gesture recognition models, and further prove that the network is lightweight, feasible and practical method.