Towards Using Generative AI for Facilitating Image Creation in Spatial Augmented Reality
Yongquan 'Owen' Hu, Wen Song Hu, Aaron John Quigley · 2023
The intersection of Generative Artificial Intelligence (GenAI) and Spatial Augmented Reality (SAR) constitutes a relatively unexplored field. The effectiveness of general-purpose generative AI models in numerous distinct scenarios and tasks remains wanting. Addressing this predicament, our paper concentrates on image generation in SAR contexts. We present the Contrastive Learning Integrated with Depth Perceptual Fusion (CLIPF) method, a refined text-plus-image-to-image generative technique, specifically devised to overcome projection-specific hurdles typically inherent in standard diffusion models. Additionally, to substantiate the efficacy of our approach, we construct a system through a cooperative amalgamation of software components, such as ChatGPT, and hardware devices, including the Intel RealSense D435 Depth Camera, thereby fostering an innovative interaction for projection content creation based on natural language. Comprehensive evaluation results validate our approach’s capacity to markedly enhance projection image quality. In summation, we illuminate potential utility avenues for our proposed approach, emphasizing its diverse prospective contributions towards enriching future SAR experiences.