Communication-Efficient Decentralized Cooperative Learning
Taha Toghani · Rice Research Repository (Rice University) · 2023
In this thesis, we study the problem of distributed cooperative learning, where a group of agents seeks to agree on a set of hypotheses that best describes a sequence of private observations. In the scenario where the set of hypotheses is large, we propose a belief update rule where agents share compressed (either sparse or quantized) beliefs with an arbitrary positive compression rate. Our algorithm leverages a unified communication rule that enables agents to access wide-ranging compression operators as black-box modules. We prove the almost sure asymptotic convergence of beliefs on the set of optimal hypotheses. We show a non-asymptotic, explicit, and linear concentration rate in probability of the beliefs on the optimal hypothesis set. The simulation results show that the number of transmitted bits can be reduced to 5-10% of the non-compressed method in the studied scenarios. We also study the social learning problem with compressed communications in the presence of adversarial agents, i.e., Byzantine. We propose a robust update rule where, under appropriate assumptions, non-faulty agents can reach a consensus on the true state of the world by sharing arbitrarily compressed messages over the network. We show almost sure asymptotic convergence of the beliefs of non-faulty agents around the optimal hypothesis and provide numerical evidence for the communication efficiency and robustness of the proposed algorithm.