FedFA: Federated Learning With Feature Anchors to Align Features and Classifiers for Heterogeneous Data
Tailin Zhou, Jun Zhang, Danny H. K. Tsang · IEEE Transactions on Mobile Computing · 2023
Federated learning allows multiple clients to collaboratively train a model without exchanging their data, thus preserving data privacy. Unfortunately, it suffers significant performance degradation due to heterogeneous data at clients. Common solutions involve designing an auxiliary loss to regularize weight divergence or feature inconsistency during local training. However, we discover that these approaches fall short of the expected performance because they ignore the existence of avicious cyclebetween feature inconsistency and classifier divergence across clients. Thisvicious cyclecauses client models to be updated in inconsistent feature spaces with more diverged classifiers. To break thevicious cycle, we propose a novel framework namedFederated learning withFeatureAnchors(FedFA). FedFA utilizes feature anchors to align features and calibrate classifiers across clients simultaneously. This enables client models to be updated in a shared feature space with consistent classifiers during local training. Theoretically, we analyze the non-convex convergence rate of FedFA. We also demonstrate that the integration of feature alignment and classifier calibration in FedFA brings avirtuous cyclebetween feature and classifier updates, which breaks thevicious cycleexisting in current approaches. Extensive experiments show that FedFA significantly outperforms existing approaches on various classification datasets under label distribution skew and feature distribution skew.