The Generalization Error of Reduced Rank Regression in Bayesian Estimation
Miki Aoyagi, Sumio Watanabe · 2004
Reduced rank regression, or a three-layer neural net-work with linear hidden units, is an important research area, because this method picks up the essential infor-mation from examples of input-output pairs. However, those models are non-regular learning machines. Its generalization error had been left unknown because of its singularities in the parameter space. In this pa-per, we introduce a new computational technique of recursive blowing-ups for densingularization of a learn-ing machine, and compute explicitly the main term in the asymptotic form of the stochastic complexity in the case of the reduced rank regression models. 1.