Improving Immersive Experiences in Telecommunication with Motion Parallax
Cha Zhang, Dinei A. F. Florencio, Zhengyou Zhang · 2011
Human sensorial perception of the surrounding environment is very intricate, and only partially understood. Visual depth perception, for example, is often attributed to stereo vision. However, if we close one eye it becomes immediately obvious how much depth information we acquire from other mechanisms. One of these mechanisms is motion parallax, i.e., the fact that the relative apparent positions of objects change when we move our viewpoint. Traditional 3DTV systems provide carefully edited stereo video, but often lack the capability of rendering any motion parallax. Similar to the visual experience, our audio experience is also affected by our body motion. Indeed, besides our two ears, we also capture a richness of information about the environment by (unconsciously) moving our heads to sample the sound field around us. This brings a rich experience, giving us much information about the sound sources and the environment. To produce a truly immersive experience, telecommunication systems will have to exploit the richness of this perception. Video will need to supply not only stereo vision, but motion parallax as well. Audio will have to reproduce the full richness of sound fields. In other words, the reproduced environment (be it a natural or synthetic environment) need to be consistent between audio and video, and need to provide the parallax sensorial perception that we are used to get in real-world environments.