Stochastic Complexity and Newton Diagram
Keisuke Yamazaki, Miki Aoyagi, Sumio Watanabe · 2004
Many singular learning machines such as neural networks and mixture models are used in the information engineering field. In spite of their wide range applications, their mathematical foundation of analysis is not yet constructed because of the singularities in the parameter space. In recent years, we developed the algebraic geometrical method that shows the relation between the efficiency in Bayesian estimation and the singularities. In this paper, we propose a new mathematical method to analyze singular learning machines based on the Newton diagram and toric deformation. Using the proposed method, we obtain the exact value of the asymptotic stochastic complexity, which is a criterion of the model selection, in a mixture of binomial distributions. 1.