Mapping a Virtual View to the Physical World to Guide the Completion of Complex Task Sequences
Bach Tran, Shivakumar Sastry · 2019 IEEE International Systems Conference (SysCon) · 2019
The upkeep of systems that are designed to operate over long periods of time requires the precise execution of complex sequences of tasks. Such maintenance activities are expensive and require large investments in training costs. We present an approach that leverages Augmented Reality (AR) to guide the completion of such complex task sequences. This system presents real-time system-status and guidance to operators to complete the task sequences. One of the critical requirements in such a system is the ability to annotate objects and artifacts of the physical world when they are viewed in an AR device. Instead of using computationally intensive approaches based on computer vision, our novel approach relies on a CAD model of the engineered system. In our approach, we transform the gaze point of the user in the virtual world to corresponding coordinates in a 3D CAD model. The high-resolution details of in the 3D CAD model allows us to extract and present appropriate annotations. In the future, this approach can be significantly enhanced by designed intelligent methods to automatically generate the AR content necessary to guide the completion of complex task sequences.