Distributed Neural Learning Algorithms for Multiagent Reinforcement Learning
Pengcheng Dai, Hongzhe Liu, Wenwu Yu, He Wang · IEEE Internet of Things Journal · 2023
In this article, the fully distributed neural learning algorithms by neural network approximation for networked multiagent reinforcement learning (NMARL) are studied. To tackle the convergence analysis of methods in NMARL with tremendous state-action space, most of the existing distributed algorithms are designed by linear function approximation, which however would fall into a situation of poor expression. To conquer such limitation, the distributed neural learning algorithms are developed by using a novel neural network approximation that bridges the theory and practice of deep NMARL (DMARL). Specifically, inspired by the overparametrization method for minimizing mean-squared projected bellman error (MSPBE), the distributed neural learning algorithms with population semigradients and stochastic semigradients are respectively, proposed to solve the NMARL problem. Furthermore, the convergence of the proposed algorithms are strictly given by employing the overparametrization method to establish the approximate stationary point of MSPBE to characterize the algorithms toward the global optimum. Finally, some numerical simulations demonstrate the effectiveness of the distributed neural learning algorithms.