From Spin Glasses to Learning of Neural Networks
E. E. Perepelkin, Boris Iosifovich Sadovnikov, Наталья Германовна Иноземцева, Roman A. Rudamenko, A. A. Tarelkin, P. Sysoev, R. V. Polyakova, M. B. Sadovnikova · Physics of Particles and Nuclei · 2022
Abstract— The conceptual basics of spin glass theory are reviewed. A description of the mathematical apparatus developed for spin glasses and the model of the restricted Boltzmann machine (RBM) is presented. Optimization of the RBM learning algorithm using nongradient methods is explored. A method to extract the learning algorithm hyperparameter, temperature, has been described and used. Critical phenomena in the RBM—entropy crisis, and difference between the temperatures of the learning sample creation and processing—are studied.