Utilising Transformers for American Sign Language Fingerspelling Recognition
Jamie Pinnington, Amina Souag, M A Hannan Bin Azhar · 2024
This paper presents a machine learning approach to American Sign Language (ASL) fingerspelling recognition using Transformer models. Addressing the challenges of high variability in hand shapes, movement, and signing speed, the study utilises the new ASL Fingerspelling Recognition Corpus and the CRISP-DM methodology to develop and evaluate a proof-of-concept model. The model achieved a mean Levenshtein distance of 4.7, corresponding to an error rate of 16.37%. This research demonstrates the feasibility of using advanced AI techniques to enhance accessibility for the deaf and hard of hearing communities by translating ASL fingerspelling into text.