Algebraic Analysis for Non-regular Learning Machines
Sumio Watanabe · 1999
Hierarchical learning machines are non-regular and non-identifiable statistical models, whose true parameter sets are analytic sets with singularities. Using algebraic analysis, we rigorously prove that the stochastic complexity of a non-identifiable learning machine is asymptotically equal to # 1 log n - (m 1 - 1) log log n + const., where n is the number of training samples. Moreover we show that the rational number # 1 and the integer m 1 can be algorithmically calculated using resolution of singularities in algebraic geometry. Also we obtain inequalities 0 < # 1 # d/2 and 1 # m 1 # d, where d is the number of parameters. 1 Introduction Hierarchical learning machines such as multi-layer perceptrons, radial basis functions, and normal mixtures are non-regular and non-identifiable learning machines. If the true distribution is almost contained in a learning model, then the set of true parameters is not one point but an analytic variety [4][9][3][10]. This paper establishes...