Application Research of Neural Network Architecture Search in Big Data Analysis
Hancong Huangfu · 2024
With the advent of the big data era, the rapid growth of data scale and complexity poses a great challenge to traditional data analysis methods. Neural Architecture Search, as an automated machine learning technique, significantly improves model performance and development efficiency by automatically designing and optimizing neural network structures. This paper systematically discusses the application of NAS in big data analysis. Firstly, the basic principle of NAS and its application in image recognition and natural language processing are reviewed. Then, combined with the characteristics of big data analysis, the advantages and challenges of NAS in dealing with massive data, high-dimensional characteristics and complex relationships are analyzed. Experiments in this paper show that NAS can automatically find the optimal neural network architecture when processing large-scale data sets, which significantly improves the accuracy and generalization ability of the model. In addition, we propose an improved NAS algorithm, which effectively improves the search efficiency by introducing multi-objective optimization strategies and reinforcement learning mechanisms. Finally, this paper summarizes the potential application prospects of NAS in big data analysis, and prospects the future research direction.