Weight space structure and analysis using a finite replica number in the Ising perceptron
Tomoyuki Obuchi, Tomoyuki Obuchi, Yoshiyuki Kabashima, Yoshiyuki Kabashima · Tokyo Tech Research Repository (Tokyo Institute of Technology) · 2016
The weight space of the Ising perceptron, in which a set of random patterns is stored, is examined using the generating function of the partition function ϕ(n) = (1/N)log[Zn] as the dimension of the weight vector N tends to infinity, where Z is the partition function and represents the configurational average. We utilize ϕ(n) for two purposes, depending on the value of the ratio α = M/N, where M is the number of random patterns. For ααs, on the other hand, ϕ(n) is used to assess the rate function of a small probability that a given set of random patterns is atypically separable by the Ising perceptrons. We show that the analyticity of the rate function changes at α = αGD = 1.245..., which implies that the dominant configuration of the atypically separable patterns exhibits a phase transition at this critical ratio. Extensive numerical experiments are conducted to support the theoretical predictions.