Selfie Sign Language Recognition with Convolutional Neural Networks
P. V. V. Kishore, G. Anantha Rao, E. Kiran Kumar, M. Teja Kiran Kumar, D. Anil Kumar · International Journal of Intelligent Systems and Applications · 2018
Extraction of complex head and hand movements along with their constantly changing shapes for recognition of sign language is considered a difficult problem in computer vision.This paper proposes the recognition of Indian sign language gestures using a powerful artificial intelligence tool, convolutional neural networks (CNN).Selfie mode continuous sign language video is the capture method used in this work, where a hearing-impaired person can operate the Sign language recognition (SLR) mobile application independently.Due to non-availability of datasets on mobile selfie sign language, we initiated to create the dataset with five different subjects performing 200 signs in 5 different viewing angles under various background environments.Each sign occupied for 60 frames or images in a video.CNN training is performed with 3 different sample sizes, each consisting of multiple sets of subjects and viewing angles.The remaining 2 samples are used for testing the trained CNN.Different CNN architectures were designed and tested with our selfie sign language data to obtain better accuracy in recognition.We achieved 92.88 % recognition rate compared to other classifier models reported on the same dataset.