Task Guided Multimodal Object Search for Service Robots via Reinforcement Learning
X L Li, Guohui Tian, Yongcheng Cui · IEEE Transactions on Industrial Electronics · 2025
Object search is a key capability for embodied intelligence in robots. Its core objective is to enable robots to navigate autonomously to specified target objects using environmental information. However, in household environments, existing methods often face numerous challenges, such as frequent search attempts and lengthy exploration paths, which significantly reduce search efficiency. In response to these issues, this article proposes an innovative task-guided multimodal object search method. By integrating the sequence of task objects, our method guides the agent’s search behavior and shortens its exploration path. Furthermore, it extracts environmental visual information and object semantic information, and utilizes graph convolutional networks to capture the relational information between objects. Through the multihead attention mechanism, it achieves a deep fusion of multimodal information, further enhancing the search efficiency for target objects. The experimental results demonstrate that our proposed method outperforms existing techniques on standard benchmarks. To further validate its effectiveness, we have successfully deployed and tested our model in the AI2THOR simulation environment and the TIAGo real robot platform.