Interact360: Interactive Identity-driven Text to 360° Panorama Generation
Zeyu Cai, Zhelong Huang, Zheng Xu, Yexin Liu, Chao Liu, Zeyu Wang, Lin Wang · 2024
360° images offer an immersive and realistic visual experience for the emerging field, such as virtual tourism, particularly when users dream of recording their sweet moments without physically visiting a place. In this case, it is valuable to generate the 360° scene of a place (e.g., seaside or mountainside) while allowing the users to generate their portraits given a taken face photo, which can be naturally harmonized with the generated 360° scene and their garment can be freely changed. In light of this, we propose a novel Interactive Identity-driven 360° Panorama Generation (Interact360) approach that produces vivid 360° panoramas featuring human portraits based on user inputs and commands, which encompass both the user’s identity and the scene text description. Interact360 consists of three interactive components: prompt refinement module, identity-driven 2D portrait generation module, and 360° panorama generation module. Our approach enables users to modify and iterate on each component according to their individualized and customized requirements. We conduct both objective and subjective analysis to evaluate the effectiveness of our approach. The quantitative and qualitative results for the portrait image generation demonstrate the superiority of our method. Moreover, user studies provide empirical evidence of our method’s effectiveness, demonstrating its strong interactivity and capacity to meet user requirements during the generation process, ultimately yielding satisfactory results for users. Our method highlights the potential benefits of panorama generation, particularly in addressing the personalized and customized needs of users.