ESMPC: An Efficient Neural Network Training Framework for Secure Two- and Three-Party Computation
Hongwei Yang, Juncheng Li, Meng Hao, Weizhe Zhang, Hui Xin He, Jinghao Zhao, Lichunxi Yang, Zhixiang Qin · ACM Transactions on Architecture and Code Optimization · 2025
In the era of big data, privacy-preserving deep neural network (DNN) training has emerged as a critical research area. Secure Multi-Party Computation (SMPC) has become a key technique for enabling collaborative model training while safeguarding data confidentiality. However, the high communication overhead inherent in SMPC protocols poses a substantial obstacle to their practical deployment, particularly in large-scale deep learning applications. To address this challenge, we propose ESMPC, a computation–communication co-optimization framework designed to enhance communication efficiency in SMPC-based DNN training. Within ESMPC, we propose SecurePipe, a novel pipeline-parallelism strategy tailored for SMPC-based DNN training. SecurePipe effectively improves both model and data parallelism, enabling parallel computation and communication under encryption, thereby enhancing overall training throughput and significantly improving GPU utilization. To further reduce communication costs, we design three communication optimization algorithms specifically targeting both linear and non-linear operations. Additionally, we propose two protocol-level optimizations that substantially lower communication overhead during secure training. Comprehensive experimental evaluations validate the effectiveness of ESMPC. In two-party computation settings, ESMPC reduces the training time of AlexNet by over 30%, and achieves more than 45% reduction for FCNN, LeNet, and ViT. Similarly, in three-party computation settings, the training time for all four models is reduced by over 45%, demonstrating the scalability and efficiency of the proposed framework in real-world SMPC training environments.