Design of big data processing system optimization based on deep learning
Huanqin Wu, Fulin Li · 2023
In the process of optimizing the big data management system, the previous intelligent methods cannot be effectively calculated. This paper first introduces the research status of parallel optimization for Convolutional neural network model training, and explains some shortcomings. Aiming at the problems still existing in the parallel optimization of existing Convolutional neural network model training, an optimization processing method oriented to Convolutional neural network is proposed. The convolutional neural network model is optimized based on the distributed parallel processing framework. The results show that the comprehensiveness and rationality of the deep learning algorithm are greater than 83%. By using a multi parameter server, the gradient calculation parallelism is improved, the communication delay loss in parallel parameter update is reduced, and data preloading is adopted to reduce the data reading time, Improve the efficiency of network model training.