Intelligent Signs Language Understanding with Autonomous Landmarks for E-learning Context
Muhammad Jamil Hussain, Ahmad Shaoor · 2022 19th International Bhurban Conference on Applied Sciences and Technology (IBCAST) · 2022
Human sign language understanding has become very important task nowadays, which is attracting interest of researchers. In this research work, an efficient sign language recognition system has been presented. The system is based on extraction of landmarks. The Mediapipe and OpenCV (Open Computer Vision) are used for extraction of landmarks. The extracted landmarks are further utilized to generate new type of features. The machine learning (ML) based classifiers have been proposed to classify the hand signs from the extracted features. The random forest has been selected as a base classifier. It presented accuracy rates of 98.76% over ISL (Irish sign language) dataset and 98.68% over ASL (American Sign Language) dataset. Other ML-based classifiers are also selected to work with new efficient features such as support vector machines (SVM), k-nearest neighbors (KNN), decision trees and naive Bayes. The results of these classifiers have also been compared. The base classifier performance is very efficient when compared to others and it works in real time (15 to 30 processed frames per second).