A New Approach for Arabic Sign Language Recognition (ArSLR)
Adham Mohamed Farouk, Abdelrahman Mohamed Zenhom, Eman Mahmoud Abdelaleem, Sara Ahmed Fadel, Khloud Khaled Attia Elsayed, Mohamed Saeed Mohammed, Roula Hani Hassan, Howida A. Shedeed · 2024
This paper presents a new approach for Arabic sign language (ArSL) recognition, employing advanced feature extraction and machine learning techniques. Using Media-Pipe, a robust framework for real-time hand and pose detection, key features are extracted from static and dynamic sign language videos. For static sign language recognition, we integrated TensorFlow Lite, which achieves an impressive accuracy of 99.1% with a CNN model and 99.5% with a Support Vector Machine (SVM) model. For dynamic sign language, we implemented a Long Short-Term Memory (LSTM) model within TensorFlow, which attained an accuracy of 93%. Enhanced performance reached 98% with a hybrid CNN-LSTM model. Our approach addresses the limitations of existing methods by eliminating the need for specialized equipment and ensuring high performance on mobile devices. Another main contribution of this research is the developing of our own dataset for static and dynamic models for Arabic Sign language. The developed static dataset is for 27 Alphabet classes (1000 images per class) and 1000 images for each word of 10 words. For the Dynamic dataset, 60 videos were cap-tured for each class. To automate the process of data col-lection and feature extraction, an automated function was developed to capture 100 images within 3 seconds. Also, a script utilizing OpenCV and Media-Pipe was implemented to capture and process video frames and extracts key points essential for model training.