Cooperative Pursuit of UAV Cluster Based on Graph Embedding Reinforcement Learning
Wanchun Guo, Wujie Xie, Wenhan Dong, Lei He · 2021
Aiming at the cooperative behavior decision-making problem of UAV cluster cooperative pursuit of escaping UAVs, a cooperative pursuit target decision-making process model based on Dec-POMDP is established.Aiming at the uncertain and partially observable scenes caused by the limited perception and communication ability of UAV cluster system state, a dynamic graph embedding method is proposed.The cluster state embedded in the graph is represented as the network input under the AC framework. Through information fusion, the individuals in the cluster system can perceive the information of the surrounding UAVs, so as to produce the global situation.Based on the idea of centralized evaluation and distributed execution, a multi-agent strategy gradient method for double delay depth determination based on empirical playback region reconstruction is proposed.This method can be effectively combined with the graph embedding method to represent the state of cluster system. The above method is applied to the target pursuit of UAV cluster, and the learning process has good convergence.