Self-Organizing Hierarchical Topology in Peer-to-Peer Federated Learning: Strategies for Scalability, Robustness, and Non-lID Data
Yoshihiko Ito, Hideya Ochiai, Hiroshi Esaki · 2023
Peer-to-Peer Federated Learning (P2P-FL) is a frame-work in which nodes communicate with each other through P2P communication to perform distributed machine learning. In this paper, our focus is on simultaneously improving learning accuracy and reducing model aggregation costs in large-scale P2P-FL environments. Additionally, we ensure resilience in the face of dynamic topology changes in a P2P network and mitigate the effects of Non-lID data, considering learning in unbalanced data and data bias. We propose a Hierarchical Topology (HT) that aggregates models in stages by dividing nodes into groups, and a grouping strategy, Group Selection Algorithm (GSA), that classifies nodes with similar data into the same group. Our experiments confirmed that the core factor in HT is the similarity of training data among nodes within the same group. By meeting this condition, we demonstrated that it was possible to achieve better learning accuracy while reducing aggregation costs compared to traditional topologies. Importantly, our proposed topology shows potential in mitigating the impact of Non-lID data. We also observe that our topology maintains robustness during the dynamic process of nodes joining or leaving.