Learning Complicated Navigation Skills from Limited Experience via Augmenting Offline Datasets
Zhiqiang Wang, Yuan Chen, Jianmin Ji · 2023
Deep reinforcement learning has yielded remarkable results in the field of robot navigation. Most of the existing RL-based methods tend to train the navigation policy with (1) a simulation environment in which the agent interacts and collects experience iteratively, (2) a shaped reward function that defines how a robot should reach the goal while avoiding collisions with obstacles. However, these methods suffer from several challenges, including the difficulty of generalizing the trained model to real-world scenarios, sub-optimal risk due to reward-shaping, and the inefficiency of data utilization. In this paper, we address these challenges by introducing the State & Goal-Relabel techniques based on Hindsight Experience Replay (HER), enabling the robot not only to learn success from failure but also to learn more complicated navigation skills from simple tasks. Instead of training only with limited real experiences, our approach aims to generate pseudo-experiences by relabeling both the local observation and target pose, cleverly improving the scale and quality of dataset, and using target-driven style to train a model with solid generalization ability. With experiences collected just in simple environments, our approach outperforms or performs comparably to classical and other learning-based methods, and generalizes well in physical environments.