Investigation of Boltzmann-Gibbs learning engines: high dimensional inference in mean-field theory and optimization of deep networks with finite resources
Manzan, Gianluca · AMS Dottorato Institutional Doctoral Theses Repository (University of Bologna) · 2025
This thesis investigates the inference properties of Boltzmann-Gibbs learning engines, focusing on their ability to learn from data and applying this understanding to optimization problems. The study focuses on high-dimensional inference in Restricted Boltzmann Machines (RBMs). These models rely on inferring structures from data as a prerequisite to leverage their generative capabilities. Central to this work is the teacher-student paradigm, where a teacher network generates datasets analyzed by a student network. Learning performance is evaluated in Bayes-optimal and mismatched regimes. In low-temperature settings, the student network effectively learns through memorization. Conversely, high-temperature datasets induce a modern signal retrieval (sR) phase, where the student aggregates partial information from noisy inputs. These findings are validated for different RBMs architectures and for the Hopfield model case. Inference capabilities are further enhanced by introducing ferromagnetic coupling among replicated Hopfield networks. This coupling significantly expands the sR region beyond what can be achieved by modifying unit priors. However, as the number of coupled student networks grows, the sensitivity of the learning region size diminishes, confirming that aligned replicas enhance learning efficiency without allowing for improbable weight regularizations. Finally, we explore energy-efficient training by optimizing the classification performance of deep neural networks (DNNs) under finite resource constraints, addressing a critical challenge in sustainable machine learning and satellite technology. By optimizing the distribution of neurons within hidden layers, the robustness of DNNs against radiation noise is improved. Insights drawn from Boltzmann Machines inform the identification of optimal architectures based on thermodynamic parameters, such as form factors and inverse temperatures. Experimental validation across multiple datasets demonstrates measurable improvements in network robustness. Together, these results provide a unified perspective on the learning performance of RBMs and related models. They also contribute to advancing sustainable technology applications by enabling robust neural network designs that perform efficiently under resource constraints.