Synchronization of musical sound and visual images: Issues of empirical and practical significance in multimedia development
Scott D. Lipscomb · The Journal of the Acoustical Society of America · 1998
In recent years, there has been a significant number of investigations into the interaction between auditory and visual systems in a multi-modal context, e.g., film, animation, and multimedia software. The author will report results of his own research, confirming a subject preference for alignment of audio-visual (AV) accent structures, i.e., combinations resulting in perceived AV synchronization. A revised version of a model of Film Music Perception [Lipscomb and Kendall (1994)] will be presented, explicating a dynamic relationship between stimulus complexity and significance of AV synchronization in the determination of subject preference ratings. Using these findings as a starting point, the author will illustrate the practical importance of musical sound and its temporal placement within a multimedia context. Multimedia software and technologically enhanced course materials created by the author will provide examples of the effect of combining musical sound and visual images. The presentation will illustrate that the combinatorial effect of such multimedia presentations far surpasses the effect of either auditory or visual stimuli in isolation, resulting in an improved aesthetic experience for the audience and an enhanced learning environment for the student.