Web-based Signing of English Text
Zexi Dai, Sathiamoorthy Manoharan, Ross Millar, Xinfeng Ye · 2022
Deaf people use sign language to communicate. There is a misconception that deaf people can read the text written in their surrounding spoken language easily. In fact, sign languages have linguistic structures quite different from their surrounding spoken languages. Therefore, a surrounding spoken language is foreign to most deaf people. Hence, the best way to help deaf people understand written text is to convert the text to signs in sign language. This paper presents a system that translates English text to American Sign Language (ASL) gloss and uses an avatar to show the signs. The system has a database that stores the sign pose data for various descriptions in sign gloss. First, the system uses a transformer-based model to translate English to ASL. Then, it looks up the sign pose data database to find the sign pose data for the descriptions in the sign gloss. The sign pose data is used to generate the animations of the signs by an avatar. The system provides a web-based interface, making it widely accessible across multiple platforms. Experiments show that the translation from text to sign language gloss achieves a good accuracy.