Learning Diverse and Efficient Goal-reaching Policies for Robot Motion Planning
Han-Cheng Yao, Chi-Kai Ho, Chung‐Ta King · 2023
In robot motion planning, providing multiple diverse trajectories to guide a robot to reach a given goal is important for the resilience of the robot to environmental changes. Recent advances in reinforcement learning (RL) have shown promising results in enabling the agents to learn diverse trajectories in motion planning. The majority of such works employed a two-part reward system, consisting of a goal-reaching reward and a diversity reward. For the former, distance rewards are normally adopted for their rich signals to guide the agents closer to the goal. The problem is that distance rewards change constantly as the agents progress towards the goal and, when combined with the diversity rewards, may interfere with the latter, causing poor learning results. In this paper, we propose to use binary rewards as the goal-reaching rewards. Binary rewards are static and predictable, allowing the diversity rewards to remain stable. The problem of sparsity of binary rewards is resolved with Hindsight Experience Replay (HER). Experimental results show that the proposed method enables the agents to learn efficient goal-reaching policies that are more diverse and robust to resist environmental changes compared to the prior approaches.