PoSE: Suppressing Perceptual Noise in Embodied Agents for Enhanced Semantic Navigation
Benhui Zhuang, Chunhong Zhang, Zheng Hu · IEEE Robotics and Automation Letters · 2023
Embodied agents navigating unknown environments face the challenge of optimizing exploration based on semantic information. Conventional methods, reliant on collected data or pre-defined rules, have limitations in scalability and applicability, while pretrained language models based methods focusing on textual modality encounter perceptual noise, which affects decision-making. To mitigate these problems, this paper presents Prompt-based Vision Context Semantic Exploration (PoSE), an innovative method that leverages prior knowledge from vision-language models (VLMs) to suppress perceptual noise. Through its unique design of prompts that use existential logic, PoSE reduces misidentifications of target objects within the observed environment. It also introduces a unique exploration map to translate target locations into exploration coordinates. The proposed methodology is evaluated on the ALFRED benchmark, demonstrating improved performance against previous rule-based and task-specific data-driven exploration policies. Furthermore, PoSE's semantic exploration policy offers enhancements over pretrained language-model based exploration methods that focus on text modality, showcasing its effectiveness and generality.