Real-time synthesis of optical lip patterns from vowel sounds
Norman P. Erber · The Journal of the Acoustical Society of America · 1977
An analog circuit has been constructed to convert speech (vowels) to Lissajous-type oscilloscope patterns in real time. The method is like that described by D. W. Boston [Br. J. Audiol. 7, 95–101 (1973)], but includes important circuit modifications to improve naturalness of lip outline and articulatory change. The speech signal receives pre-emphasis resembling ear canal resonance and is split into two channels. Two voltages, one proportional to 200–1000-Hz energy (F1) and the other proportional to 200–3000-Hz energy (F2 in front vowels; F1 in back vowels), are generated to control a multiplier and divider, respectively, which vary levels of sinusoidal signals to form the height and width of the lip outline. Resulting patterns are comparable to lip shapes of vowels in natural speech. Vowel-confusion matrices obtained by lipreading the oscilloscope display (for speech input) are similar to those obtained by lipreading the talker's face directly. The device has been employed in studies of auditory-visual speech perception, and has potential as a telephone communication aid for the deaf. [Work supported by NINCDS.]