Matrix-update for accelerated on-line learning in multilayer neural networks
Siegfried Bös · Journal of Physics A Mathematical and General · 1998
In this letter we re-examine the emergence of plateaus or symmetric phases in on-line learning of a committee machine. We propose a simple matrix-update in order to avoid the symmetric phase. Simulations show that the length of the plateaus can be considerably decreased. Annealing of the learning rate can then be applied much earlier, making on-line learning more effective.