Static Arabic Sign Language Recognition in Real Time Using Machine Learning and MediaPipe
Lamis Ali Hussein, Ziad Saeed Mohammed · 2024
Sign language is a form of visual communication used by individuals who are deaf or hard of hearing to communicate. This visual language relies on gestures, handshapes, facial expressions, and body movements to convey meaning rather than spoken words. Therefore, to improve the lives of deaf or hard of hearing people in the Arab community, a more comfortable approach to learning and working must be developed. This paper presents an interactive computer vision-based system for recognizing static hand gestures (letters and numbers) in Arabic sign language in real-time. The MediaPipe framework is used to extract features (hand landmarks) from each image, and a support vector machine to recognize the static gesture inputted in front of the camera and then translate it into its equivalent text and voice, an approach developed to bridge the communication gap between deaf and hearing people. Experiments were conducted using HopeArSL, a large dataset that collected 12,000 images of 40 Arabic sign language gestures. This approach's experiments showed 100% and 99.94% training accuracy for numbers and alphabet letters, respectively. The accuracy and average response time in real-time for numbers and alphabet letters were (98.08%, 1.43 ms) and (99.09%, 7.48 ms), respectively.