Finding correspondence between visual and auditory events based on perceptual grouping laws across different modalities
Jinji Chen, Toshiharu Mukai, Yoshinori Takeuchi, Hiroaki Kudo, T. Yamamura, Noboru Ohnishi · 2002
A human being understands the environment by integrating information obtained by sight, hearing and touch. To integrate information across different senses, a human being must find the correspondence of events observed by different senses. The paper seeks to relate the audio-visual events caused by more than one movement according to general physical laws without object-specific knowledge. As corresponding cues, we use Gestalt's grouping law; simultaneity of the occurrence of sound and change in movement, similarity of time variation between sound and movement, etc. We conducted experiments in a real environment and obtained satisfactory results showing the effectiveness of the proposed method.