Preventing local minima by decorrelation
Chong Ho Lee · 2002
In this paper, a new method of identifying and eliminating local minima which commonly appear in recurrent neural networks is presented. The energy surface of the neural network is re-sculptured by decorrelating the spurious states during learning process so as to remove the local minima. The spurious states are identified by applying a stationary condition to the set of admissible states. The stability verification is done efficiently by a specially designed parallel network. The decorrelation is employed to only those predetermined spurious states. As the result of this "unlearning", the correct retrieval rate as well as the storage capacity of the dynamic associative memory is improved.