Evaluation of a computer aided 3D lip sync instructional model using virtual reality objects
Arumugam Rathinavelu, Harshavarthan Thiagarajan, S. Savithri · 2006
Lip sync model is one of the aspects in computer facial animation. To create realistic lip sync model, a facial animation system needs extremely smooth lip motion with the deformation of the lips synchronized with the audio portion of speech. A deformable, parametric model of the lips was developed to achieve the desired effect. In order to create realistic speech animation, the articulatory modeling of the lip alone is insufficient. The other major articulators such as the tongue and jaw must also be considered. This lip sync model was initially developed by using polygonal model and blended key shape techniques and then parameterized by using 36 control points. The data for lip sync model was collected from video image and magnetic resonance imaging (MRI) techniques. The articulatory movements of our lip sync model were presented along with virtual reality (VR) objects in an interactive multimedia (IMM) interface. This IMM interface was used to teach small vocabulary of hearing impaired (HI) children. Virtual reality objects used to increase the cognitive process within HI children. The bilabial speech sounds were differentiated by using appropriate visual cues. Control panel was developed to present articulatory movements at different speed. For this study, six hearing impaired children were selected between the ages 4 and 7 and they were trained for 10 hours across 2 weeks on 18 meaningful words. The intelligibility of hearing impaired children was experimented to find out their performance in articulation and in memory retention. The results indicated that 65-75 % of given words were articulated well and 75-85 % of words were identified by all children. 1.