A Gesture Recognition Method Based on Edge-Terminal Collaboration and Transfer Learning
Chao Wang, Liang Zhang, Tao Wang, Rui Guo, Feng Zhu · 2024
Current gesture recognition systems encounter challenges in terms of accuracy, response time, efficiency, and susceptibility to background interference. To address these challenges, this study introduces a high-performance method based on hand keypoints and transformer learning, integrating edge-terminal collaboration and transfer architecture. The method employs a hand keypoint CNN model to extract features from RGB gesture images, capturing 3D coordinates and finger structure. Additionally, the method extends recognition to real-time video and incorporates a multi-head attention mechanism with relative position encoding to enhance accuracy with gesture sequences. Evaluation on the Hagrid dataset demonstrates that the method achieves an RGB mode accuracy of 96.49%, outperforming MobileNetV3 by 3.55%. Notably, it relies solely on an optical camera and minimal 3D data, avoiding the need for additional sensors or complex setups. Meeting real-time requirements, it provides a natural interaction method and seamlessly integrates into various devices due to its flexibility and scalability.