Real-Time Continuous Sign Language Classification using Ensemble of Windows
Rinki Gupta, Nitu Jha · 2020
Sign language recognition using wearable sensors will help in bridging the communication gap between the deaf people who use sign language and the hearing people, who do not understand signing. In this paper, classification of continuously signed sentences in real-time, using multi-modal wearable sensors is considered. Real-time classification requires segmentation of signals on which classification is carried out. Segmentation of signals with fixed window length is convenient, however, not suitable for addressing the variation in signal as static hand postures and dynamic hand motions are performed during signing. Hence, an ensemble of classifiers learned over features extracted from windows of different lengths is proposed to enhance the accuracy with which the continuously signed sentences may be classified. Results for 11 sentences signed from the Indian sign language using data from surface electromyogram and accelerometers placed on both the hands of the signers are presented. The effect of using different classifiers and different window lengths is also studied. The duration required to classify a sentence from the recorded signal segment indicates the real-time nature of the algorithm. The proposed ensemble approach classifies the sentences with higher accuracy as compared to using a single classifier learned on features extracted from fixed-duration windows.