Lightweight Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM) for Dynamic Hand Gesture Recognition
Leander Lizo, Jheanel Espiritu Estrada · 2024
Dynamic hand gesture estimation in the form of writing letters in the air, entails recognition of hands in conjunction to the relative position at set timestamp to an end timestamp. The researchers’ investigation into CNN and LSTM architecture revealed the effectiveness of this hybrid approach for spatial and temporal feature extraction, respectively. With the excessive resources needed to run convolutional neural networks, the research identifies a gap and tries to solve this problem by creating a less process intensive convolutional neural network that will be able to run on low specification and create a dynamic hand gesture system. From that standpoint, the researchers created variants to improve the model of MobileNetV1 as it gained the highest accuracy among various CNN models tested on the air writing dynamic hand gesture dataset. 1) Layer reduction removes one subset of separable convolutions from the base MobileNetV1. 2) Adding layers to improve generalization and optimization of the feature extractor. The improved model for feature extraction were evaluated using the basic performance metrics such as accuracy, precision, recall, and f1 score, on which both garnered good results. Experiment 1 reduced the size of MobileNetV1 to 8MB but has reduced performance to 0.93. Experiment 2 resulted to an increase of performance by almost 0.01 (0.99 accuracy) and increasing the size with a negligible 0.01 MB (12.33MB total size, with an increase of 4096 parameters). It was concluded that both of the modification on MobileNetV1 are necessary upgrades of the model and both can be used depending on the size and performance prioritization as it provides faster inference, reduced model size, and compatibility with devices with low computational capacity.