Point Cloud Stream on Spatial Mixed Reality - Toward Telepresence in Architectural Field
Tomohiro Fukuda, Yuehan Zhu, Nobuyoshi Yabuki · eCAADe proceedings · 2018
In remote meetings that involve the study of buildings and cities, sharing three-dimensional (3D) virtual spatial of buildings and cities is just as necessary as sharing the appearances and voices of meeting participants. Because of this, system development and pilot projects have attempted to share 3D virtual models via the internet in real-time but is still insufficient compared with face-to-face meeting. Therefore, this research explores the applicability of a spatial mixed reality (MR) system that displays point cloud streams to realize 3D remote meeting in architecture and urban fields. MR is a new technology that enables 3D presentations of various information, combining the physical and virtual worlds. One MR method is telepresence, which is expected to give people a way to communicate remotely as if face to face in a realistic way. We first developed a MR system named PcsMR (Point cloud stream on mixed reality) to display point cloud streams. The PcsMR system's operation consists of generating and transferring a point cloud stream and then rendering a point cloud stream using MR. The PcsMR acquired the point cloud stream in real-time using Kinect for Windows v2 and transferred it to Microsoft HoloLens, which uses optical see-through MR. Then we constructed two prototypes based on PcsMR and carried out pilot projects. Through observing the experiments, application possibilities for architecture and urban fields are found in meetings and communications that share real-time 3D objects and include the movement of remote participants and objects. The proposed method was evaluated feasible and effective.