Desingularization and the Generalization Error of Reduced Rank Regression in Bayesian Estimation
Miki Aoyagi, Sumio Watanabe · 2004
Reduced rank regression, or a three-layer neural network with linear hidden units, is an important learning machine because it extracts the essential information from training samples. However, its generalization error had been left unknown because of its singularities in the parameter space. In this paper, we propose a new method of recursive blowing-ups for densingularization of a learning machine. By applying it to the reduced rank approximation, we show the eectiveness of the method and clarify the asymptotic generalization error of the reduced rank regression.