A Stochastic Algorithm Based on Cable-stayed Bridges Model for Stochastic Optimization in Machine Learning
Linguang Tai, Feng Shan, Ruibin Wang, Yulun Wu, Xinjian Li, Qinmin Yao, Gang Yan, Chen Ouyang, Gonglin Yuan · Journal of Physics Conference Series · 2023
Abstract This paper takes the economic cost of road design as the objective function to study the optimization of road construction cost. It is well-know that stochastic optimization is widely regarded as one of the most important and difficult problems in optimization and machine learning. It is noted that various methods for stochastic optimization problems are sensitive to the learning rate (step-size), which is usually underestimated. Therefore, two new stochastic conjugate gradient algorithms are proposed in this paper. We adapt two modified line search techniques to set the stochastic optimization. One uses the modified Armijo line search technology, the other uses the modified weak Wolfe-Powell line search technology. The new search direction has the properties of sufficient descent and trust region, which makes the establishment of global convergence of the two algorithms very understandable and clear. Finally, numerical experiments show that the two new stochastic conjugate gradient methods are competitive compared with other classical methods.