On the correlation between articulatory and acoustic data
Jintao Jiang, Abeer A. Alwan, Patricia Keating, Lynne E. Bernstein, Edward T. Auer · The Journal of the Acoustical Society of America · 2000
In J. Acoust. Soc. Am. 107, 2904 (2000), we investigated the correlations between external facial movements, tongue movements, and speech acoustics using a database of consonant–vowel syllables spoken by one female and two male talkers. The intelligibility of the talkers, based on visual information, was judged by hearing-impaired individuals. In this study, we extend the analysis to two sentences and a fourth talker (female); each sentence was repeated five times. Two of the talkers (one male and one female) had high-intelligibility ratings while the other two had low ratings. A Qualisys (optical motion capture system) and an EMA system were used to characterize orofacial and tongue movements, respectively. The articulatory and acoustic data streams were recorded simultaneously and aligned. Acoustic features were represented by line spectral pairs. A multilinear regression technique was applied to quantify the correlation between the data. A universal estimator was obtained with training data, using a jack-knife approach, and then applied to the test data. Correlations results were speaker dependent, and, on average, the highest correlations were between the optical and EMA data. Detailed results, including a comparison between the results for CVs and sentences, and implications will be discussed. [Work supported in part by NSF KDI award 9996088.]