High-Throughput Privacy-Preserving GRU Network with Homomorphic Encryption
Zeyu Wang, Makoto Ikeda · 2023
The deep learning technique has been applied in a wide range of applications. The development of cloud computing further expands the application scenarios of deep learning but brings privacy issue. Homomorphic Encryption (HE), as a cryptographic solution, has attracted much attention in recent years. While some research applies HE on forward neural networks (FNNs), the research on recurrent neural networks (RNN) is still rare because the deep recurrent operation is hard to implement with HE. In this paper, we propose the first gated recurrent unit (GRU) network on HE scheme without bootstrapping, which is very expensive that takes 73% calculation time in the former encrypted GRU design. We introduce several techniques to reduce the multiplicative depth to 8 per recurrent step and allow 7 recurrent steps in our model. Processing long input sequences is also available by a rearranging method to control the recurrent steps less than 7. The testing results show that there is nearly no accuracy degradation between our encrypted GRU and the original unencrypted model. Also, we highly improve the throughput by a packing strategy. On MNIST dataset, we achieve 98.6% accuracy, which is totally the same as the unencrypted result. The throughput is 439 images/hour, which is 360 times higher than the former design. Furthermore, we test more complex natural language processing (NLP) task, which is a dominant application of RNNs. Our encrypted GRU network achieves 90.0% accuracy on AG-news dataset, almost the same as the unencrypted GRU model of 90.3%. The results indicate that the encrypted GRU design can provide accurate, efficient and privacy-preserving predictions.