Breaking the Quadratic Communication Overhead of Secure Multi-Party Neural Network Training
Xingyu Lu, Başak Güler · 2023
Privacy-preserving machine learning has achieved exciting breakthroughs for collaboratively training machine learning models under strong information-theoretic privacy guarantees. Despite the recent advances, communication bottleneck still remains as a major challenge against scalability to large neural networks. To address this challenge, in this work we introduce CLOVER, the first multi-party neural network training framework with linear communication complexity, significantly improving over the quadratic state-of-the-art, under strong end-to-end information-theoretic privacy guarantees. CLOVER builds on a novel degree reduction mechanism with linear communication complexity, termed Double Lagrange Coding, for coded computing. While providing strong multi-round information-theoretic privacy guarantees, CLOVER achieves equal adversary tolerance, resilience to user dropouts, and model accuracy as the state-of-the-art, while significantly cutting down the communication overhead. In doing so, CLOVER addresses a key technical challenge in collaborative neural network training, paving the way for large-scale privacy-aware deep learning applications.