Synthesizing Novel Spaces for Remote Telepresence Experiences
Mohammad Keshavarzi, Michael Zollhöfer, Allen Y. Yang, Patrick Peluse, Luisa Caldas · 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · 2022
The emerging field of remote telepresence via spatial computing has opened many exciting opportunities for next-generation computer-mediated-communication platforms. Such techniques enable users to mutually engage in a wide spectrum of applications, previously not possible in 2D screen-based communication methods. Yet, cali-brating and finding a mutual environment compatible with all remote participant's physical environment is considered a challenging task. In this paper, we elaborate on the mutual space finding problem and provide a high-level introduction of our proposed novel Mutual Scene Synthesis (MSS) system. The MSS system takes the partici-pants' surrounding environment as input, and synthesizes a virtual scene that corresponds to the functional features of all participants' physical spaces. By combining a function optimization module with a deep-learning conditional scene augmentation process, the MSS can generate a scene compatible to all participants of a remote telepresence scenario. By performing early comparative user studies via the MatterPort3D dataset, we evaluate the effectiveness of our system and show our proposed MSS approach can be a promising re-search direction for facilitating contextualized telepresence systems for next-generation spatial computing platforms.