Automatic Spotting of Sign and Fingerspelling for Continuous Sign Language Recognition
Hee-Deok Yang, LeeSeongWhan · Jeongbo gwahaghoe nonmunji. so'peuteuweeo mich eung'yong · 2011
Signs are dynamic gestures discriminated by continuous hand motions and hand configurations, while fingerspellings are combinations of continuous hand configurations. Sign language spotting is the task of detection and recognition of signs and fingerspellings in a signed utterance. The internal structures of signs and fingerspellings differ significantly. Therefore, it is difficult to spot signs and fingerspellings simultaneously. In this paper, a novel method for spotting signs and fingerspellings is proposed. It can distinguish signs, fingerspellings and non-sign patterns. This is achieved through a hierarchical framework consisting of three steps: (1) Candidate segments of signs and fingerspellings are discriminated using a two-layer Conditional Random Field (CRF). (2) Hand shapes of segmented signs and fingerspellings are verified using BoostMap embeddings. (3) The motions of fingerspellings are verified in order to distinguish those which have similar hand shapes and different hand motions. Experiments demonstrate that the proposed method can spot signs and fingerspellings from utterance data at rates of 83% and 78%, respectively.