Multi-UAV Formation Transformation Based on Improved Heuristically-Accelerated Reinforcement Learning
Yanbing Xiao, Yingzhou Zhang, Yuxin Sun, Junyan Qian · 2019
Addressing the shortcomings of the classical reinforcement learning algorithms in solving the problem of multi-UAV formation transformation, such as large consumption of computing resources and slow convergence speed, this paper introduces heuristics and uses back propagation algorithms to construct a heuristic function required for multi-UAV formation convergence. When designing the action selection function of reinforcement learning algorithms, the idea of simulated annealing is also introduced, which enables the algorithm to fully explore the combination of convergence actions in the early stage so as to jump out of the local optimal trap, while ensuring the convergence of the algorithm in the later stage. The data of the final simulation experiment show that the improved heuristically-accelerated reinforcement learning algorithms can effectively reduce the consumption of UAV computing resources, improve the convergence speed the convergence quality when dealing with the problem of multi-UAV formation transformation.