Performance Evaluation of Feature Fusion Approaches for Indian Sign Language Recognition System
B. V. Poornima, S Srinath, S. Rashmi, R Rakshitha · Indian Journal of Science and Technology · 2023
Objectives: To recognize and analyze the Indian sign language (ISL) gestures in simple background using various features and Machine learning classifiers. Methods: a) Data pre-processing: Contour matching approach for hand region segmentation b) Feature extraction: Local features like gradient & key point descriptors were extracted using HOG, SIFT, SURF, LBP, FAST, feature fusion is done by concatenating features of HOG with LBP, SIFT with FAST, BOVW model with SURF. c) Model development: SVM, Random Forest, Logistic regression, Naïve Bayes were trained on large dataset and was experimented with hyper parameter tuning. The experiment was performed on 2 standard datasets which consist of alpha numerals (A-Z & 1-9) in simple black background. Findings: Our research demonstrates that our model consistently achieved a remarkable 100% accuracy rate when utilizing feature fusion techniques on both of the datasets employed in this study. The research findings underscore the need to consider a more comprehensive approach for gesture recognition. Relying solely on distinct features extracted from a single algorithm is shown to be insufficient in addressing the challenges posed by the diverse nature of sign shapes, varying illumination conditions, and different orientations. This emphasizes the importance of exploring hybrid or multi-algorithmic strategies to enhance the accuracy and robustness of gesture recognition systems. Novelty: This research introduces a novel perspective on ISL gesture recognition by emphasizing gestures against a simple black background. The innovative application of feature fusion, combining various elements like hand shape and orientation enhances accuracy. The inclusion of hand segmentation using contour matching algorithm and experimentation on benchmark data adds another layer of novelty, highlighting practical applicability. The experimental results show that SVM has given better results when used with different combinations of feature extractors. Keywords: ISL, Feature Fusion, Keypoint Descriptors, HOG, SIFT, SURF, LBP, FAST