A Distributed Framework for Deep Reinforcement Learning by Consensus
Bo Liu, Shuang Zhu, Peng Sun, Qisheng Huang, Zhengtao Ding · 2023
This paper proposes a distributed training framework for deep reinforcement learning algorithms to address large-scale problems with privacy protection. First, we design a hierarchical decentralized communication topology with a server to alleviate the heavy burden on the central server. By pushing the model updating and data storage to the edge side, it not only unleashes the computing potential of the terminal devices but also shortens the response time. Second, the consensus algorithm is applied for all agents over the decentralized topology to approach each other, where each agent only requires neighboring information to achieve consensus. Then, the distributed training framework for deep Q-networks is designed based on the consensus algorithm, which consists of local learning and global consensus. Finally, the simulation demonstrates that the proposed distributed learning deep Q-networks shows better performance than its centralized learning counterpart.