Collaborative Work with Volumetric Data Using Augmented Reality Videoconferencing
István Barakonyi, Tamer Fahmy, Dieter Schmalstieg, Karin Kosina · International Symposium on Mixed and Augmented Reality · 2003
The Augmented Reality Videoconferencing System is a novel remote collaboration tool combining a desktop-based AR system and a videoconferencing module. The novelty of our system is the combination of these tools i.e. superim- posing AR applications on live video background displaying the parties' real environment, thus merging the advantages of videoconferencing (natural face-to-face com- munication) and AR (interaction with distributed virtual ob- jects using tangible physical artifacts). We demonstrate the system's collaborative features with a volume rendering ap- plication that allows users to display and examine volumet- ric data simultaneously and to highlight or explore slices of the volume by manipulating an optical marker as a cutting plane interaction device. We developed an AR videoconferencing tool that runs AR applications superimposed on live video background. The system merges the advantages of videoconferencing and AR, combining natural face-to-face communication with the capability to interact with distributed virtual ob- jects using tangible physical artifacts. The par- ties' real environment is recorded with a video camera, al- lowing them to see each other and enhance their communi- cation with non-verbal cues. Virtual objects, i.e. the ference material, are bound to physical optical markers that can be freely positioned in the user's environment. Our application is built on the Studierstube collaborative AR platform (5). Studierstube supports multi-user interac- tion on various configurations including our desktop-based setup. The interaction and object manipulation is done with the Personal Interaction Panel (PIP), a two-handed AR in- terface provided by the Studierstube platform. The PIP con- sists of two tracked, handheld physical props for each user: a panel augmented with virtual interaction widgets (buttons, sliders, etc.) and a pen for user input (i.e. clicking, drag- ging etc.) on the widgets. Tracking data for the interaction props can come from either optical markers or keyboard commands. Each party sees two application windows on the screen: one representing the local user and the other representing the remote party. The application is currently limited to two collaborators. The local user's application window displays the virtual objects, which are interaction props and the conference material objects, overlaid over the local video. The position and orientation are calcu- lated from optical markers that are extracted from the local video sequence. The application window representing the remote party displays the remote video sequence, which is encoded and compressed on the remote computer and sent over the network via the videoconferencing module. At the same time, the remote user's tracking data is extracted and streamed to the local machine using multicast or TCP/IP connections. Position and orientation information of the optical markers have already been processed on the remote client's machine while the actual rendering and overlay over the video data is done on the local machine. Our system is superior to a video-only solution since the image quality is significantly better. Tracking data calcula- tions are not duplicated. The tracking information is more precise, since it is extracted from the higher-quality local video, and it is possible to interact with the virtual objects. Advantages of our system over an interactive applica- tion sharing approach are the much higher speed and lower bandwidth, and the fact that no extension module for proper handling of real-time video is needed.