Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning
Girum Fitihamlak Ejigu, Ye Lin Tun, Apurba Adhikary, Sun Moo Kang, Choong Seon Hong · 2023
All involved clients are guaranteed data privacy in a collaborative machine learning environment via Federated Learning. The lack of generalization in local client models brought on by data heterogeneity, however, is one of Federated Learning ’s major challenges that lead to slow model convergence and communication latency. In this study, we use contrastive learning techniques that have been shown effective in both centralized and federated learning environments. For local models to have a stronger capacity for generalization, we suggest adopting contrastive loss at the model and data scales. Utilizing the CIFAR10 and CIFAR-100 datasets, we assess and contrast our suggested strategy with other industry standard approaches.