An Effective Approach for Resource‐Constrained Edge Devices in Federated Learning
Jun Wen, Xiusheng Li, Yupeng Chen, Xiaoli Li, Hang Mao · International Journal of Intelligent Systems · 2024
Federated learning (FL) is a novel approach to privacy‐preserving machine learning, enabling remote devices to collaborate on model training without exchanging data among clients. However, it faces several challenges, including limited client‐side processing capabilities and non‐IID data distributions. To address these challenges, we propose a partitioned FL architecture that a large CNN is divided into smaller networks, which train concurrently with other clients. Within a cluster, multiple clients concurrently train the ensemble model. The Jensen–Shannon divergence quantifies the similarity of predictions across submodels. To address discrepancies in model parameters between local and global models caused by data distribution, we propose an ensemble learning method that integrates a penalty term into the local model’s loss calculation, thereby ensuring synchronization. This method amalgamates predictions and losses across multiple submodels, effectively mitigating accuracy loss during the integration process. Extensive experiments with various Dirichlet parameters demonstrate that our system achieves accelerated convergence and enhanced performance on the CIFAR‐10 and CIFAR‐100 image classification tasks while remaining robust to partial participation, diverse datasets, and numerous clients. On the CIFAR‐10 dataset, our method outperforms FedAvg, FedProx, and SplitFed by 6%–8%; in contrast, it outperforms them by 12%–18% on CIFAR‐100.