Análise de física estatística em modelos de aprendizado de máquina
Rodrigo Soares Veiga · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · 2022
This thesis presents three main contributions to the understanding of machine learning models by making use of statistical physics tools. First, we investigate the possible relation between the renormalisation group and restricted Boltzmann machines trained with two-dimensional ferromagnetic Ising data, pointing out possible misleadings in preliminary proposals to explicitly construct this bridge. Secondly, we examine the convergence behaviour of stochastic gradient descent in high-dimensional two-layer neural networks. By building up on classic statistical physics approaches and extending them to a broad range of learning rate, time scales, and hidden layer width, we construct a phase diagram describing the various learning scenarios arising in the high-dimensional setting. We also discuss the trade-off between learning rate and hidden layer width, which has been crucial in the recent mean-field theories. Thirdly, we study both Bayes-optimal and empirical risk minimization generalisation errors of the multi-class teacher-student perceptron. We characterise a first-order phase transition arising in the Bayes-optimal performance for Rademacher teacher weights and observe that, for Gaussian teachers, regularised cross-entropy minimisation can yield to close-to-optimal performance.