SignEnd: An Indian Sign Language Assistant
Leafia Dias, Ketaki Keluskar, Anviksha Dixit, Krunal Doshi, Mrinmoyee Mukherjee, Joanne Gomes · 2022 IEEE Region 10 Symposium (TENSYMP) · 2022
Communication is an essential day-to-day activity that human society thrives on. Indian Sign Language (ISL) is one form of oral communication among the deaf-mute community in India. As the general public usually tends to be unaware of this form of interaction, daily conversations are strenuous for a deaf-mute person. Previously created systems focus on detecting alpha-numeric signs pertaining to users with five fingers solely. This paper describes an ISL system that can recognize the alpha-numeric hand signs of its users with five and six fingers and translate them into their corresponding text equivalences. A custom dataset is created that explicitly tailors to these requirements. Additionally, this system can convert entered text (letter, number, word, sentence) into corresponding sign equivalences. Sign-to-Text conversion is achieved by using Mediapipe-Hands Machine Learning (ML) model to detect hand signs for deaf-mute people with five fingers. Similarly, an Object Detection Application Programming Interface (API) is implemented to detect hand signs for users with six fingers. Presently, the proposed system has an average accuracy of 90 percent.