Privacy Preservation Linear Regression Scheme with Applications
Eric Z. Shen · Research Square · 2022
Abstract Linear regression is a basic and widely used machine learning algorithm, and the training of linear regression models usually depends on a large amount of data, but in reality, the data sets are usually held by different users and contain users' privacy information, so when multiple users want to pool a large amount of data to train a better model, it will inevitably involve users' privacy. Homomorphic encryption, as a privacy protection technique, can effectively solve the privacy leakage problem in computing. A new privacy-preserving linear regression scheme based on a hybrid iterative approach is designed for the scenario where the data set is horizontally distributed over two users, combined with the CKKS homomorphic encryption technique. In the second phase, a secure two-sided fast descent protocol is designed, which is based on the Jacobi iterative algorithm and can effectively compensate for the ineffectiveness of the gradient descent method in practical applications and speed up the collection of the model, thus reducing the computational cost and communication loss of the scheme. This reduces the computational cost and communication loss of the scheme, and protects the data privacy of both users while training the linear regression model efficiently. The efficiency, communication loss, and security of the scheme are analyzed, and the scheme is implemented in C++ and applied to real data sets. Extensive experimental results show that this scheme can efficiently solve linear regression problems with large feature sizes, and the relative error of the decidable coefficients is less than 0.001, which indicates that the obtained privacy-preserving linear regression model is close to the model obtained directly on the plaintext data in real data sets, and can meet the practical application requirements in specific scenarios.