Data Privacy-Preserving and Communication Efficient Federated Multilinear Compressed Learning
Di Xiao, Zhuyan Yang, Maolan Zhang, Lvjun Chen · 2024
Federated Learning (FL) has received widespread attention as a collaborative learning paradigm. Clients can collaboratively train a global model with server by uploading parameters instead of sharing the raw data, which guarantees the basic data privacy. However, recent research has highlighted that sensitive information can be inferred from shared updates or gradients, resulting in serious privacy leakage. Moreover, transmitting the updates or gradients can result in communication bottlenecks. In order to solve the privacy and communication problems, we propose a federated multilinear compressed learning framework (FedMCL), which considers the tensor structure of the data and performs multidimensional compression on the client’s raw data through multilinear compressed learning. Compared with vector-based compressed learning, it has better learning performance on multi-dimensional data. We generate proxy images from the measurement domain for training, which effectively defends against gradient inversion attacks. In addition, we introduce low-rank approximation method to compress model updates and reduce the communication overhead by transmitting small matrices instead of the original model updates. Experimental results show that our scheme can resist gradient inversion attacks under different compression rates while obtaining good classification performance, which has advantages in terms of privacy, performance and communication.