Statistical Mechanics of Node-Perturbation Learning for Nonlinear Perceptron
Kazuyuki Hara, Kentaro Katahira, Kazuo Okanoya, Masato Okada · Journal of the Physical Society of Japan · 2013
Node-perturbation learning is a type of statistical gradient descent algorithm that can be applied to problems where the objective function is not explicitly formulated, including reinforcement learning. Node-perturbation learning with \(M\) linear perceptrons has previously been analyzed using the methods of statistical mechanics. It was shown that cross-talk noise, which originates from the error of the other outputs, increases the generalization error as the number of outputs increases. On the other hand, a nonlinear perceptron has several advantages over a linear perceptron, such as the ability to use nonlinear outputs, learnability, storage capacity, and so forth. However, node-perturbation for a nonlinear perceptron has yet to be analyzed theoretically. In this paper, we derive a learning rule of node-perturbation learning for a nonlinear perceptron within the framework of REINFORCE learning and analyze the learning behavior by using statistical mechanical methods. From the results, we found that the signal and cross-talk terms of the order parameter \(Q\) have different forms for a nonlinear perceptron. Moreover, the increase in the generalization error with increasing number of outputs is less than for a linear perceptron.