Efficiently Assemble Normalization Layers and Regularization for Federated Domain Generalization
Khiem Le, Van Long Ho, Cuong Do, Danh Le-Phuoc, Kok‐Seng Wong · 2024
Domain shift is a formidable issue in Machine Learning that causes a model to suffer from performance degradation when tested on unseen domains. Federated Domain Gener-alization (FedDG) attempts to train a global model using collaborative clients in a privacy-preserving manner that can generalize well to unseen clients possibly with domain shift. However, most existing FedDG methods either cause additional privacy risks of data leakage or induce signifi-cant costs in client communication and computation, which are major concerns in the Federated Learning paradigm. To circumvent these challenges, here we introduce a novel architectural method for FedDG, namely gPerXAN11https://github.com/lhkhiem28/gPerXAN, which relies on a normalization scheme working with a guiding regularizer: In particular, we carefully design Personalized eXplicitly Assembled Normalization to enforce client mod-els selectively filtering domain-specific features that are bi-ased towards local data while retaining discrimination of those features. Then, we incorporate a simple yet effec-tive regularizer to guide these models in directly capturing domain-invariant representations that the global model's classifier can leverage. Extensive experimental results on two benchmark datasets, i.e., PACS and Office-Home, and a real-world medical dataset, Camelyon17, indicate that our proposed method outperforms other existing methods in ad-dressing this particular problem.