LGF-SLR: Hand Local–Global Fusion Network for Skeleton-Based Sign Language Recognition
Qing Hong Gao, Meiqi Zhang, Zhaojie Ju · IEEE Sensors Journal · 2025
For those who are hard of hearing, sign language recognition (SLR) technology can offer a more practical and effective means of communication. A skeleton-based SLR system has gained prominence and has been employed in practical settings due to its advantages in generalization, computing accuracy, antiinterference, and privacy protection. It is unfortunate that this method’s low recognition accuracy is frequently caused by inadequate hand-skeletal information. In order to recognize sign language, the multistream fusion method is also frequently employed. This approach, however, frequently uses artificial weight assignment, which has been identified as having drawbacks, including subjectivity and inaccuracy. This research proposes a Hand local-global fusion network for skeleton-based SLR (LGF-SLR), which deals with the aforementioned difficulties. To the greatest extent possible, the skeleton information is preserved while it is extracted and inputted independently into the network’s global feature streams (upper body) and local feature streams (left and right hands). Concurrently, the multistream fusion component incorporates Bayesian optimization, which automatically assigns weights to disparate prediction knots and mitigates the influence of manually assigned weights on trial outcomes. Last, the AUTSL sign language dataset, the WLASL2000 American sign language (ASL) dataset, and the SLR500 Chinese sign language dataset are employed to evaluate the efficacy of the LGF-SLR. The findings indicate that the LGF-SLR exhibits superior accuracy compared to existing methods, with accuracy of 95.93%, 52.74%, and 98.42%, respectively. Code is available athttps://github.com/MeiqiZhang7/LGF-SLR.