Private Read-Update-Write With Controllable Information Leakage for Storage-Efficient Federated Learning With Top r Sparsification
Sajani Vithana, Şennur Ulukuş · IEEE Transactions on Information Theory · 2023
In federated learning (FL), a machine learning (ML) model is collectively trained by a large number of users, using their private data in their local devices. With toprsparsification in FL, the users only upload the most significantrfraction of updates, and download only the most significantr’ fraction of parameters in order to reduce the communication cost. However, the values and the indices of the sparse updates and parameters leak information about the users’ private data. In this work, we consider an FL setting whereNnon-colluding databases store the model to be trained, from which the users download and update sparse parameters privately, without revealing the values of the updates/parameters or their indices to the databases. We propose four schemes with different properties that are based on cross subspace alignment (CSA) and permutation techniques, to perform this task while achieving the minimum communication costs within the scope of CSA, and show that the information theoretic privacy of both the values and the positions of the sparse updates/parameters can be guaranteed. This is achieved at a considerable storage cost, though. To alleviate this, we generalize the schemes in such a way that the storage cost is reduced at the expense of a certain amount of information leakage, using a model segmentation mechanism. In general, we provide the trade-off between the communication cost, storage cost and information leakage in private FL with toprsparsification.