HANDTALK: Adaptive and Interactive Self-Learning System for Deaf and Dumb School Students in Sri Lanka
Kobigan Krishnananthan, Aroosha Rasheed, Ratheeskan Thadchaneswaramorthy, Kumaran Sathiyavarathan, Samantha Thelijjagoda, Karthiga Rajendran · 2023
In response to the unique educational needs of hearing- and speech-impaired students in Sri Lanka, this paper introduces a web-based application designed to bridge this gap. The system empowers these students with the ability to learn sign language independently, through tutoring and self-revision. It leverages a range of technologies, including sign language detection via camera, sentiment analysis, machine translation, grammar checking, progress tracking, question prediction, recommendations, reinforcement learning, and activity monitoring. One pivotal feature is the revision system, which allows students to pose questions based on Cambridge O-level academic content. The system generates accurate responses, which are further utilized to offer personalized question-and-answer recommendations. The revision system boasts an impressive accuracy rate of 90%, ensuring that students receive precise and relevant answers to their queries. The second key component is a tutoring system, enabling both hearing- and speech-impaired individuals and students of all ages to acquire sign language skills. The system includes a monitoring component, which accurately tracks student behavior. Both the tutoring and monitoring systems achieve high accuracy rates of 90% and 80%, respectively. These robust systems foster effective learning, personalized support, and learner engagement for a diverse student population in Sri Lanka.