Dynamic Gesture Recognition Based On Multimodal Fusion Model
Juan Fang, Chao Xu, Chao Wang, Hua Li · 2021
Aiming at the problem that the RGB modal is susceptible to illumination, skin color, complex background, and lack of motion information, resulting in low recognition accuracy. A dynamic gesture recognition algorithm based on the joint ResNeXt-3D-CBAM model is proposed. The algorithm uses a 3D convolution kernel to replace the traditional 2D convolution kernel in the ResNeXt network, and effectively extracts the spatiotemporal features of dynamic gestures in RGB, Depth and optical flow data. Then use the method of multi-model fusion to predict dynamic gestures. Experimental results show that the method proposed in this paper can recognize dynamic gestures more accurately. Compared with the traditional single-modal information, the accuracy of dynamic gesture recognition is improved, which verifies the feasibility and superiority of the proposed method.