New evidence for audiovisual speech scene analysis: Low level interaction between auditory streaming and visual cues in speech perception
Frédéric Berthommier, Jean‐Luc Schwartz · The Journal of the Acoustical Society of America · 2012
We have proposed in the last years that there could exist a level of audiovisual binding (audiovisual speech scene analysis) previously to audiovisual fusion and speech comprehension. In this paper, we propose a new paradigm in which a competition exists between a non-speech auditory stream and an audiovisual speech stream for incorporating the pre-voicing component (PVC) excised from a /b/. The auditory stream is composed of low intensity PVCs at a regular rhythm of about 1Hz, interleaved with /pa/ syllables at about 0.3Hz, forming the speech stream. The listeners' tasks is an online forced choice between /p/ and /b/, which relies on the level of capture of a PVC within the speech stream (hence perceived as /b/) or within the auditory stream (speech being hence perceived as /pa/). The regular PVC stream has the tendency to capture the common PVC. Surprisingly, the vision of lips movements enhances this capture effect, and the phonetic fusion between the common PVC and the /p/ is disfavored. To characterize this effect, two supplementary experiments have been carried out, suggesting this effect is (at least partly) speech specific. This supports our proposal for an audiovisual speech binding mechanism.