A Real-time Sign Language Recognition System using MediaPipe and Random Forest Classifier combining with Text-to-Speech Technique
Shyam Reddy, Subhash Mondal, P. M. G. Bashir Asdaque, K Vamsi Krishna · 2025
This research study introduces a real-time sign language detection system utilizing hand gestures, specifically developed to bridge the communication gap for deaf and hard-of-hearing individuals. The system employs advanced computer vision techniques, utilizing MediaPipe for accurate hand tracking and landmark detection, extracting key features such as fingertip and palm positions. These features are normalized to address variations in hand size and orientation, ensuring consistent input for classification. A Random Forest Classifier (RFC), trained on a labeled dataset of gestures, processes the data to achieve reliable recognition. The system offers dual feedback mechanisms: real-time visual text overlays on live video streams and Text-to-Speech (TTS) output, making it intuitive and user-friendly. Iterative training and out-of-bag error monitoring enhance model performance, ensuring seamless functionality across devices ranging from mobile phones to larger systems. Motivated by the need for inclusivity, this solution enables effortless communication in both social and professional contexts. By leveraging AI-driven approaches, the system provides an accessible and efficient tool for promoting understanding and interaction, advancing assistive technologies for deaf and hard-of-hearing communities. This work represents a significant step toward creating more inclusive communication solutions.