Sign language translation based on syntactic and semantic analysis
Masahiro Abe, Hiroshi Sakou, Hirohiko Sagawa · Systems and Computers in Japan · 1994
Abstract The prototype sign language translation system described herein interprets Japanese sign language to verbal Japanese language. Since sign language has a unique morphology and syntax, the sign language translation system requires linguistic translation after recognition of the input symbols. One of the main difficulties in translation is a result of the insufficiency of syntactic markers in the input information. In Japanese sign language, input is represented as a sequence of important words. In our method, semantic dependency analysis based on case frame grammar is carried out to complement the lack of information such as postpositions. Then, rule‐based rephrasing is executed to obtain natural verbal language. This scheme contributes both flexibility and ease of maintenance to the translation system. A prototype sign language translation system was implemented and tested using 123 sentences extracted from a sign language textbook. Experimental results indicate that 76 percent of the sentences can be translated correctly if they have no recognition error in input sentence. However, if a recognition error occurs, the experiment indicates that the translation will become highly ambiguous. Therefore, the capability for rejecting incorrect sentences is very important, along with the capability for translating correct input sentences. Future issues include incorporation of unused information, such as facial expression, spatial information and contexts in sign language to improve translation accuracy.