Automatic Recognition of Auslan Finger-spelling using Hidden Markov Models
P.M.Y. Goh · 2005
In recent years, gesture recognition has received much attention from research communities. Computer vision-based gesture recognition has many potential applications in the area of human-computer interaction as well as sign language recognition. Sign languages use a combination of hand shapes, motion and locations as well as facial expressions. Finger-spelling is a manual representation of alphabet letters, which is often used where there is no sign word to correspond to a spoken word. In Australia, a sign language called Auslan is used by the deaf community and and the finger-spelling letters use two handed motion, unlike the well known finger-spelling of American Sign Language (ASL) that uses static shapes. This thesis presents the Auslan Finger-spelling Recognizer (AFR) that is a real-time system capable of recognizing signs that consists of Auslan manual alphabet letters from video sequences. The AFR system has two components: the first is the feature extraction process that extracts a combination of spatial and motion features from the images. Which classifies a sequence of features using Hidden Markov Models (HMMs). Tests using a vocabulary of twenty signed words showed the system could achieve 97% accuracy at the letter level and 88% at the word level using a finite state grammar network and embedded training.