Automatic Optimization of Hyperparameters for Deep Convolutional Neural Networks: Grid Search Enhanced with Coordinate Ascent
Qingqing Song, Shaoliang Xia, Zhen Wu · 2024
Using the same depth convolutional neural network model will result in significantly different results due to different combinations of hyperparameters. By adjusting the configuration of hyperparameters, we can enhance the performance of the model. However, hyperparameter optimization typically requires a significant amount of computational resources and time. Therefore, improving the efficiency of hyperparameter optimization is crucial. In this study, we utilized the coordinate ascent method, which only offered initial candidate values for each hyperparameter. By changing only the hyperparameter that had the greatest effect on the model in each iteration, we gradually expanded the search grid until we achieved convergence in accuracy. This method enables us to effectively and automatically find hyperparameter combinations that can improve model accuracy. The experimental results show that using the MWD dataset, the model optimized through hyperparameters achieved an accuracy of 95.71% on the validation set, and this hyperparameter combination can be considered an approximate global optimal solution. In addition, the performance of the hyperparameter combination in its neighborhood is stable, which further proves the robustness of our hyperparameter optimization strategy.