Real-time Fine-grained Gesture Recognition Based on Millimeter Wave Radar
Juan Fang, Yong Xu, Wei Liu, Li Cheng, Qun Fang, Xin Yi He · 2024
Fine-grained gesture activity recognition holds broad application prospects in smart education, assisted driving, and other fields, especially in non-contact, high-precision gesture-human interaction. Existing methods for real-time gesture recognition often require cumbersome gesture collection and operate in offline mode, not fully addressing practical issues such as gesture segmentation and recognition latency. To address these challenges, an Awa-GNN is proposed to handle sparse point cloud data and extract universal features, eliminating the need for additional collection and training, thus mitigating personalized differences among users. Moreover, by integrating bi-directional gated recurrent units (Bi-GRU) based on convolutional attention, temporal features are extracted to achieve efficient gesture recognition. Experiments demonstrate that Awa-GNN outperforms other baselines in terms of generalization and recognition accuracy on the MMGesture dataset and self-collected datasets. To validate the effectiveness of Awa-GNN, a real-time gesture recognition system named P-Gesture is constructed. The system uses a sliding window mechanism based on the state machine (SM) for automatic gesture segmentation and is deployed on the IWR1642 radar. Additionally, an end-to-end real-time application based on P-Gesture has been developed.