An Imbalanced Mean-Field Game Theoretical Large-Scale Multiagent Optimization With Constraints
Shawon Dey, Hao Xu · IEEE Transactions on Systems Man and Cybernetics Systems · 2024
A novel optimization algorithm has been developed for distributed large-scale multiagent systems (LS-MASs), specifically focusing on achieving a terminal density constraint. While the recent advancement in mean field game (MFG) offers a feasible distributed solution to address the “Curse of Dimensionality” problem, it compromises the optimality of large-scale homogeneous agents and lacks the capability to achieve arbitrary fixed terminal probability density function (PDF) constraint, especially when deviating from the normal distribution. To tackle this issue, a novel approach called the imbalanced mean-field game (Imb-MFG) theory has been designed alongside an adaptive PDF decomposition method and distributed reinforcement learning (RL) that can effectively obtain optimal solutions in LS-MAS even with fixed terminal density constraints in a distributed manner. In particular, a method based on the induction theory has been developed for estimating the parameter of the final PDF constraint, enabling the decomposition of MFG-PDF into multiple imbalanced normal distributions. Subsequently, the Imb-MFG theory is developed by integrating the decomposed multigroup MFG agents with a K-means clustering algorithm with constraint. The developed Imb-MFG approach decomposes a single PDF into multiple imbalanced normal distributions, which are combined to achieve arbitrary terminal PDF constraints. To achieve the solution of the Imb-MFG theory, a multiactor-critic-mass (M-ACM) algorithm is developed. This algorithm is developed to concurrently learn the solution for coupled Fokker-Planck–Kolmogorov (FPK) and Hamilton-Jacobi–Bellman equations. The algorithm’s convergence is ensured through the Lyapunov analysis. The effectiveness of this algorithm is validated through a simulation study.