Enhancing Real-Time Converter for Bangla Sign Language with the Integration of Dynamic Words
Jarin Tasnim Tonvi, S.M. Rashidul Hasan Nijhum, Towfiq Rahman · 2024
The system of using visual hand gestures and signals known as sign language, is used by people, who have difficulty in hearing and speaking to communicate. Bangla Sign Language (BdSL) is one of the 137 sign languages used in the globe. Despite the large community consisting of 3 million deaf and dumb, who use BdSL in real life, not many studies have been conducted on this topic. Especially, in everyday practice, sign language for Bangla words predominates BdSL. But regrettably, it is not adequately examined in research studies. So, this study suggests a system that can recognize Bangla sign language words including words, that include the motion of the hands, called dynamic words. Though the Bangla sign language dictionary declares maximum words as dynamic signs, work has yet to be done for detecting dynamic words to the best of our knowledge. Therefore, a system is built to recognize these motion-based signs. For this purpose, a dataset of dynamic signs is also developed. To comprehend and evaluate the performance of the system for sign words of varying degrees of difficulty, the dataset is divided into three distinct categories. The MediaPipe holistic approach is used instead of image processing techniques for faster recognition and minimal computation compared to usual image processing techniques to identify postures, which is a novel approach for dynamic word detection. The system is deployed using 4 machine-learning approaches- CNN, RNN, LSTM, and KNN. The performances of the proposed system are evaluated, compared, and contrasted with other methods for detecting sign language. The proposed system achieves an accuracy of 99% for static words; additionally, we evaluated the rarely explored dynamic words and got an accuracy of 80% for them. Moreover, We included some ambiguous words in addition to both static and dynamic words.