Sign Language to Text Classification using One-Shot Learning
Sourav Pal, Sourabh Ruhella, Sparsh Sinha, Anurag Goel · 2023
Sign language is utilized by deaf individuals to communicate with each other. Sign languages, simply put are signs and gestures made by hand and are currently being used by approximately 80 million people around the world. The limited knowledge of sign languages in the hearing-abled community makes it difficult for them to interact with the deaf or mute community leading to the marginalization of the latter. Sign-to-text classification is a field of research that aims to convert sign language into written text, providing a communication bridge between the deaf-mute communities and the hearing population. However, there are some limitations in the current state of the field, such as the absence of datasets that are publicly available for sign language in many languages, and most of the existing models are trained in American Sign Language (ASL) which makes it difficult for researchers to train and evaluate their methods on other sign languages. Additionally, the ASL-trained models may generalize poorly to other sign languages, as they are heavily tuned to the specific dataset of ASL signs. The current models are also not best suited to incorporate any new additions to the sign language as it requires to train the model all over again. The proposed model makes use of one-shot learning, a new-age machine learning technique that allows the model to learn and recognize using limited training data, to identify and classify hand signs and convert them into corresponding text. The ability to learn from a very small dataset makes the model easily trainable and expandable in any sign language of choice. The proposed model has been trained on ASL and Indian Sign Language (ISL) datasets and achieved accuracy of 90.03% and 91.25% respectively.