Unbiased Learning for Hierarchical Models
Masashi Sekino, Katsumi Nitta · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · 2007
It is known that overfitting occurs when a conventional statistical learning method such as maximum likelihood estimation, maximum a posteriori estimation or Bayesian estimation is applied to hierarchical models. This paper gives an explanation why overfitting occurs and propose an appropriate learning framework unbiased learning for hierarchical models. The method suggest to train the hyperparameters based on unbiased likelihood which is estimated by an appropriate information criterion. Therefore, it can say that the unbiased learning is a generalization of hyperparameters selection. Unbiased learning with several information criteria is tested by computer simulations.