A path planning method based on noisy D3QN algorithm with N-step updates
Xuefeng Pei, Lieping Zhang, Ming Zhang, Yameng Yin, Zhufei Leng, Yilin Wang, Huaquan Gan · Ain Shams Engineering Journal · 2025
To address the issues of unstable Q-value estimation and insufficient exploration during the early training stage of the Dueling Double Deep Q Network (D3QN), an N-step and Noisy D3QN algorithm is proposed. First, an N-step update strategy is designed, in which multi-step cumulative rewards replace single-step rewards. Second, learnable exploration noise is incorporated into the neural network so that each action selection of the mobile robot depends not only on the Q-value but also on stochastic perturbations, thereby enhancing exploration ability. Finally, ablation studies are conducted to quantify the incremental contributions of each component. Across multiple simulation environments with both static and dynamic obstacles, the proposed algorithm outperforms DDQN, D3QN, SAE-DDQN, and D3QN-PER in terms of average path length, average number of steps, and average traversal time. Furthermore, experiments in real-world environments verify the feasibility and robustness of the proposed method.