Research on distributed machine learning defence strategies under Byzantine attacks
Chen Jin, Xi Chen, Junyu Pu, Boyu Fan · 2024
The emergence of distributed machine learning has greatly improved the speed of machine learning model training, however, distributed machine learning is prone to be damaged by Byzantine attacks in practical applications leading to model training failure. In this paper, we investigate the security of distributed machine learning with parameter server architecture and propose a distributed machine learning strategy that can resist malicious attacks. And based on this strategy, a distributed machine learning framework is built using lightweight techniques. The strategy is designed as follows: for the case where the training node suffers from Byzantine attack, the Byzantine fault-tolerant stochastic gradient descent algorithm is used to carry out local gradient aggregation; for the case where the central node of distributed machine learning suffers from malicious attack, the consensus node is selected by using the consensus mechanism of the union chain, and the average gradient of the consensus node is used instead of the global gradient to participate in the training of the global model. The experimental environment is constructed through containerization technology to simulate the distributed machine learning network with high fidelity and low resource consumption. The experimental results show that the anti-Byzantine attack strategy proposed in this paper can effectively defend against Byzantine attacks. Under the same malicious attack scenario, its model accuracy is improved by about 20% compared to the distributed machine learning framework with traditional parameter server architecture.