Development and convergence analysis of training algorithms with local learning rate adaptation
George D. Magoulas, Vassilis P. Plagianakos, Michael N. Vrahatis · 2000
A new theorem for the development and convergence analysis of supervised training algorithms with an adaptive learning rate for each weight is presented. Based on this theoretical result, a strategy is proposed to automatically adapt the search direction, as well as the step-size length along the resultant search direction. This strategy is applied to some well known local learning algorithms to investigate its effectiveness.