A Guided-to-Autonomous Policy Learning Method of Deep Reinforcement Learning in Path Planning
Wang Zhao, Ye Zhang, Haoyu Li · 2024
This study introduces a Guided-to-Autonomous Policy Learning (GAPL) method that improves the training efficiency and composite optimization of Deep Reinforcement Learning (DRL) in path planning. Under this method, firstly, we introduce the concept of guiding rewards as a reward enhancement mechanism, which, based on Rapidly-exploring Random Trees (RRT) and Artificial Potential Field (APF) algorithm, effectively addresses the challenge of training efficiency. We then propose the Guided-to-Autonomous Reward Transition (GART) model to solve the combined challenges of balancing training efficiency with composite optimization problems, which lies in the evolutionary refinement of the reward structure, initially dominated by guiding rewards, transiting progressively toward a focus on rewards that emphasize composite optimization, specifically minimizing the distance and time to the end point. Simulated experiments in static obstacle settings and mixed dynamic-static obstacle environments demonstrate that: 1) guiding rewards play a significant role in enhancing training efficiency; 2) the GAPL method yields superior composite optimization outcomes for path planning compared to conventional methods, and it effectively addresses the issue of training efficiency in conventional DRL method.