Functional analytic approach to model selection-subspace information criterion
Masashi Sugiyama · 1999
: The problem of model selection is considerably important for acquiring higher levels of generalization capability in supervised learning. In this paper, we propose a new criterion for model selection called the subspace information criterion (SIC). Computer simulations show that SIC works well even when the number of training examples is small. Keywords: supervised learning, generalization capability, model selection, Hilbert space, projection learning. 1 Introduction Supervised learning is obtaining an underlying rule from given training examples, and can be regarded as a function approximation problem. So far, many learning methods for supervised learning have been developed, including the back-propagation algorithm [4, 19], projection learning [13], Bayesian inference [10], and support vector regression [25, 21]. In these learning methods, the quality of the learning results depends heavily on the complexity of models. Here, models refer to, for example, Hilbert spaces to w...