An Improved Q-Learning Algorithm with Particle Swarm Optimization for Path Planning
Zhiyao Ma, Zhongxin Liu · 2024
This paper proposes a novel path planning method which combines improved Particle-Swarm-Optimization and Q-Learning(IPQL) to address path planning problems in complex environments. Using the global optimization capability of PSO, the initialization of the Q-table is improved, significantly accelerating the convergence of the Q-Learning algorithm and enhancing the quality of the planned paths. In the PSO optimization phase, a fitness function incorporating path length, obstacle avoidance, and smoothness is designed to generate a high-quality initial Q-table. In the Q-Learning optimization phase, an improved dynamic epsilon-greedy strategy is employed, with a dynamically changing epsilon value to balance exploration and exploitation, overcoming the issue of local optima commonly encountered in traditional Q-Learning. Experimental results demonstrate that the IPQL algorithm outperforms the traditional Q-Learning algorithm in terms of both path length and smoothness, with significantly faster convergence. Compared to the baseline IDQ algorithm, the IPQL method has a better performance in terms of path length and path smoothness. The proposed method effectively enhances the efficiency and accuracy of path planning, showing promising potential for practical applications.