Some Aspects of Influence of Nonlinearities to Storage Capacity of Neural Networks by Alternative Mean Field Theory
Algis Garliauskas · SSRN Electronic Journal · 2004
In the paper the more realistic neuronal soma and synaptic nonlinear relations and an alternative mean field theory (MFT) approach relevant for strongly interconnected systems as a cortical matter are considered. The general procedure of averaging of the quenched random states in the fully connected networks for MFT as usually is based on the Boltzmann Machine learning. But this approach requires an unrealistically large number of samples to provide a reliable performance. We suppose an alternative MFT with instead of stochastic nature of search a solution a set of large number equations with deterministic features. Of course this alternative theory will not be strictly valid for in nite number of elements. Another property of generalization is an inclusion of the additional member in the effective Hamiltonian allowing to improve the stochastic hill-climbing search of the solution not dropping into local minimum of the energy function Especially we pay an attention to increasing of neural networks retrieval capability transforming the replica symmetry model by including of different nonlinear elements Some results of experimental modeling as well as the wide discussion of neural systems storage capacity are presented.