Fully Robust Federated Submodel Learning in a Distributed Storage System
Zhusheng Wang, Şennur Ulukuş · IEEE Transactions on Information Theory · 2024
We consider the federated submodel learning (FSL) problem in a distributed storage system. In the FSL framework, the full learning model at the server side is divided into multiple submodels such that each selected client needs to download only the required submodel(s) and upload the corresponding update(s) in accordance with its local training data. The server comprises multiple independent databases and the full model is stored across these databases. A fundamental problem in FSL is to enable these multiple databases at the parameter server to collectively maintain a consistent view of the global parameters to facilitate parallel computing across distributed clients. In addition, a practically implementable FSL scheme should possess high throughput, efficient performance, fault tolerance, sufficient privacy, adequate security, certifiable stability, elastic scalability and easy usability features. To specifically resolve the fault tolerance and adequate security issues together, we propose a novel coding mechanism coined ramp secure regenerating coding (RSRC), which is a synthesis of ramp secret sharing and secure regenerating code. This coding technique matches FSL perfectly, as the system performance can be further improved when the full model is stored using RSRC in a distributed manner. By incorporating and extending available techniques cleverly, our new RSRC-based distributed FSL approach that is constructed on top of our earlier two-database FSL scheme which uses private set union (PSU), achieves all of the aforementioned important features. A complete one-round FSL process consists of: 1) an FSL-PSU phase where the union of the submodel indices to be updated by the selected clients in the current round is determined, 2) an FSL-write phase where the updated submodels are written back to the databases, and 3) additional auxiliary phases where sufficient amounts of necessary common randomness are generated at both server and client sides.