Improved Indian Regional Sign Language Recognition with Extended IRKSL Dataset
C.N. Sujatha, Prathamesh Jadi, N B Shubham, Sakshi Narayan Habib, U M Chaitanya, Padmashree Desail · 2024
Sign language serves as a vital means of communication within the deaf and hard of hearing (DHH) community, offering a unique linguistic structure and grammar separate from spoken languages. However, its intricate nature presents a substantial barrier to entry, dissuading many from mastering it. Consequently, this linguistic divide often leads to communication challenges and social isolation for members of the DHH community. To address this significant issue, deep learning techniques have emerged as a powerful tool to recognize sign gestures and convert them into corresponding text. In this paper, the Extended Indian Regional Sign Language Dataset (X-IRKSL) has been introduced, focusing specifically on the Kannada regional language. This dataset comprises a diverse range of videos, encompassing a total of 116 classes and benefiting from recordings made by experienced signers from a deaf school. Notably, this dataset is uniquely tailored to mathematical sign language, emphasizing its relevance and impact within the educational domain. The study utilizes the MediaPipe holistic framework to identify sign language gestures from keypoints within the Kannada sign language dataset-KSL. It prepares the data by preprocessing and normalizing keypoints and utilizes Bidirectional LSTM layers to model temporal aspects in a sequential manner. The model achieves an impressive accuracy of $\mathbf{9 1. 2 5 \%}$ on the test set, showcasing its effectiveness in overcoming communication obstacles for the DHH community. The results of the IRKSL paper [1] demonstrate a word-level accuracy of 71.5% on the IRKSL dataset, while our methodology on the IRKSL dataset achieves an accuracy of $\mathbf{8 4. 7 1 \%}$, indicating significant improvement and efficacy. Furthermore, this paper conducts a comparison of datasets, including the INCLUDE dataset [2], to evaluate the performance of the proposed model, providing valuable insights into the advancement of sign language recognition technologies.