FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillation
Quyang Pan, Sheng Sun, Zhiyuan Wu, Yuwei Wang, Min Liu, Bo Gao, Jingyuan Wang · 2024
Federated Edge Learning (FEL) has emerged as a promising approach for enabling edge devices to collaboratively train machine learning models while preserving data privacy. Despite its advantages, practical FEL deployment faces significant challenges stemming from device constraints and device-server interactions, necessitating heterogeneous, user-adaptive model training with limited and uncertain communication conditions. Knowledge Cache-driven Federated Learning (FedCache) is a promising architecture that enables communication-efficient and heterogeneous-aware collaborative training in edge computing scenarios. However, previous work is limited by the intrinsic nature of logits-based interactions, leading to performance bottlenecks due to the poor richness of exchanged information for on-device model optimization. To tackle this issue, we introduce FedCache 2.0, a novel personalized FEL architecture that enhances the exchange of optimization insights while delivering state-of-the-art performance with efficient communication. FedCache 2.0 incorporates the benefits of both dataset distillation and knowledge cache-driven federated learning by storing and organizing distilled data as knowledge in the server-side knowledge cache, allowing devices to periodically download and utilize personalized knowledge for local model optimization. Moreover, a device-centric cache sampling strategy is introduced to tailor transferred knowledge for individual devices within controlled communication bandwidth. Extensive experiments on five datasets covering image recognition, audio understanding, and mobile sensor data mining tasks demonstrate that (1) FedCache 2.0 significantly outperforms state-of-the-art methods regardless of model structures, data distributions, and modalities. (2) FedCache 2.0 can train splendid personalized on-device models with at least ×28.6 improvement in communication efficiency. Our code is available at https:// github.com/ poppanda/ FedCache2.0.