MIFD-Net : a hand gesture recognition model based on feature fusion of MLP and CNN
Tong Wang, Ge Song, WeiJian Ni, Qingtian Zeng · Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022) · 2023
Gesture recognition can play a crucial role in addressing the issue of Human-computer interaction. In this paper, we proposed a vision-based Multi-input fusion deep network (MIFD-Net), which consists of Multilayer Perceptron (MLP) and Convolutional Neural Networks (CNN). MIFD-Net first processes hand keypoint data and gesture images using Euclidean distance normalization (ED-Normalization) and image segmentation technologies, respectively. Then, two kinds of data are simultaneously used as input to MIFD-Net. The experimental results show that the MIFD-Net achieves an average accuracy of 99.65% on the self-built dataset in this paper and 99.10% on the NUS hand posture datasets II (NUS-II). The MIFD-Net significantly decreases its FLOPs and the number of parameters and reduces the complexity of the model while maintaining a high recognition rate compared with other gesture recognition models. The MIFD-Net can obtain high accuracy and strong robustness in different environments, lighting, and angles.