CLIP-LSTM: Fused Model for Dynamic Hand Gesture Recognition
Reena Tripathi, Bindu Verma · 2023
The computer vision field continues to find dynamic hand gesture detection to be an intriguing subject. The algorithm is unable to determine accurately whether a gesture begins or ends in a video feed therefore, recognizing dynamic hand movements in real time is challenging. Real-time dynamic hand gesture detection has several applications, and numerous researchers are working on it. In this paper, we have used the CLIP model to extract the features of hand gestures. Then the extracted features passed into the BLSTM model to classify the dynamic hand gestures. Using the CLIP model for feature extraction overcomes the problem of hand detection and tracking. The various illumination makes hand detection and tracking challenging and CLIP is used to extract the features of each video. We conduct an experiment with fewer parameters on a challenging dataset. Experimental results on the CHG and LISA dataset with 97% accuracy on CHG and 86% accuracy on LISA datasets shows that our proposed model outperforms the state-of-the-art methods (SOTA).