Path Planning Based on Deep Reinforcement Learning in Unstructured Environment
Wei Fu, Ximeng Liu, Yanbo Yang · 2024
In unstructured environments, the irregular distribution, complex shapes, and varied attributes of obstacles make it challenging to define effective reward objectives in deep reinforcement learning frameworks.Consequently, issues such as reward sparsity, reward deception, and convergence to local optima frequently arise. To address these challenges, this paper presents an enhancement strategy that combines external distance-based rewards with internal state-based incentives. Specifically, the integration of a distance-attenuated external reward strategy enables dynamic adjustment of distance rewards, thereby alleviating reward sparsity and minimizing reward spoofing, while optimizing path length through the attenuation of weight coefficients. However, as distance rewards attenuate, exploration may become insufficient, potentially leading to local optima. To counter this, a state-novelty-based internal reward strategy is introduced, encouraging exploration in areas with high obstacle density. Simulation results demonstrate that the proposed enhanced reward strategy outperforms conventional approaches in unstructured environments, significantly reducing path length and the number of turns required in planned paths.