A Real-time Multimodal Hand Gesture Recognition via 3D Convolutional Neural Network and Key Frame Extraction
Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang · 2018
In this paper, we present an approach for hand gesture recognition by 3D Convolutional Neural Network 3D_CNN and key frames extractor algorithm by the fast neural network. Typically, 3D_CNN algorithms classify hand gestures from a number of randomly sampled image sequences. In this work, key frames extracted from static video summarization based clustering method was used as input of 3D_CNN algorithm to improve classification accuracy. Because of expensive computation of video summarization, we propose a fast deep neural network SegNet based video summarization method VSUMM to learn and identify key frames in a video sequence to speed up the computational time to appropriate with a real-time system. We evaluate our proposed algorithms on publish Cambridge gestures dataset and Seven Hand Gestures SHG Dataset. We also experimentally estimate number of key frames per a video sequence in those datasets. This algorithm achieved 94.4% classification accuracy on the Cambridge gestures dataset and 77.71% classification accuracy on the SHG Dataset. The experiment results show that the proposed approach is efficient and outperforms to compare with state-of-art related researches with real-time computational efficiency.