A new information criterion for the selection of subspace models.
Masashi Sugiyama, Hidemitsu Ogawa · 2000
. The problem of model selection is considerably important for acquiring higher levels of generalization capability in supervised learning. In this paper, we propose a new criterion for model selection named the subspace information criterion (SIC). Computer simulations show that SIC works well even when the number of training examples is small. 1. Introduction Supervised learning is obtaining an underlying rule from training examples, and can be regarded as a function approximation problem. In virtually all learning methods, the quality of the learning results depends heavily on the complexity of models. The problem of model selection has been studied from various standpoints: information statistics [1, 3], Bayesian statistics [5, 2], stochastic complexity [4], and structural risk minimization [6]. Many model selection criteria devised so far use asymptotic approximation in their derivation, so they do not work well when the number of training examples is small In this paper, we pro...