Decentralized Optimal Multi-agent System Tracking Control Using Mean Field Games with Heterogeneous Agent
Zejian Zhou, Hao Xu · 2021 IEEE Conference on Control Technology and Applications (CCTA) · 2021
In this paper, a decentralized optimal tracking control problem has been studied for a large-scale multi-agent system (MAS) with heterogeneous system dynamics. Due to the agent number of large-scale MAS, the notorious “curse of dimensionality” problem has challenged the traditional MAS algorithms for decades. The emerging mean field game (MFG) theory has recently been widely adopted to generate a decentralized control method that tackles those challenges by encoding the large-scale multi-agent systems’ information into a Probability Distribution Function (PDF). However, the traditional MFG methods assume all agents are homogeneous, which is unrealistic in practical industrial applications, e.g., IoTs, etc. Therefore, a novel mean field Stackelberg game (MFSG) is formulated based on the Stackelberg game, where all the agents have been classified as two different categories where one major leader’s decision dominates the other minor agents. Moreover, a hierarchical structure that treats all minor agents as a mean field group is developed to tackle homogeneous agents’ assumptions. Then, the actor-actor-critic-critic-mass $(A^{2}C^{2}M)$ algorithm with five neural networks is designed to learn the optimal policies by solving the MFSG. The Lyapunov theory is utilized to prove the convergence of $A^{2}C^{2}M$ neural networks and the closed-loop system’s stability. Finally, series of numerical simulations are conducted to demonstrate the effectiveness of the developed method.