Generalization in Learning: Eyring Formula and Predator-prey Model
Sergei Vladimirovich Kozyrev, Ilya A. Lopatin, Alexander Nikolaevich Pechen · Lobachevskii Journal of Mathematics · 2025
Abstract In this work, generalization in machine learning is investigated using a predator-prey model. We overview biological and chemical interpretation of the stochastic gradient Langevin dynamics and generative adversarial network. For a particular example of recovering an unknown function by a polynomial of fixed degree from a set of noisy data, the predator-prey model provides better approximation compared to the gradient descent method. Further, thermodynamical arguments, in particular Eyring formula, are used to explain grokking (delayed generalization) phenomenon in machine learning.