Utilization of Fixed Structure Learning Automata for Adaptation of Learning Rate in Backpropagation Algorithm
Hamid Beigy, Mohamad Reza Meybodi, Mohammad Bagher Menhaj · Journal of Applied Sciences · 2002
Error backpropagation training algorithm (BP) which is an iterative gradient descent algorithm is a simple way to train multi layer feedforward neural networks. Despite the popularity and effectiveness of this algorithm, its convergence is extremely slow. The main objective of this paper is to incorporate an acceleration technique into the BP algorithm for achieving faster rate of convergence. By interconnection of Fixed Structure Learning Automata (FSLA) to the feedforward neural networks, we apply Learning Automata (LA) scheme for adjusting the learning rate based on the observation of random response of neural networks. The feasibility of proposed method is shown through simulations on three learning problems: Exclusive-or (XOR), approximation of function sin(x), and digit recognition. These problems are chosen because they possess different error surfaces and collectively present an environment that is suitable to determine the effect of proposed method. The simulation results show that the adaptation of learning rate using this method not only increases the convergence rate of learning but it increases the possibility of bypassing the local minima.