Dynamic gesture recognition method based on multi-scale feature fusion
Feiyue Qiu, Jian Qiang Zhou, Delong Peng · 2024
Currently, dynamic gesture classification methods based on deep learning have problems such as the inability to extract gesture features thoroughly and difficulty in accurate time series modeling of gesture features. This paper proposes a dynamic gesture recognition method based on multi-scale feature fusion to address these problems. First, dynamic gesture depth image sequences can be converted into 3D point cloud sequences to maintain the gesture’s spatial structure and shape features. Then, a two-stream neural network structure is introduced to learn local and global features of gesture point cloud data separately, and they are fused into multi-scale features. To more accurately model dynamic gesture features in time series, a BiGRU-Transformer module is constructed by adding a bidirectional gated recurrent unit to the original structure of the Transformer, into which multi-scale features are input to obtain the final recognition results. To validate the effectiveness of our method, we perform experiments on SHREC17 and DHG datasets. The method achieves the highest recognition accuracy on these two datasets compared to other methods, validating the superiority of our proposed method.