Statistical Analysis of UnidentiÞable Models and its Application to Multilayer Neural Networks
Kenji Fukumizu · 2000
This paper discusses the maximum likelihood estimator of the parametric model that has lack of identiÞability in low dimensional subsets in the parameter space. Among many statistical models with unidentiÞability, neural network models are the main concern of this paper. The unidentiÞable true parameter is formulated as a conic singularity ofthemodelembeddedinaninÞnite dimensional space of probability density functions. Following Hartigan’s idea, the likelihood ratio of the maximum likelihood estimator is described by the supremum of an empirical process over a set of functions. It has been known in some models the asymptotics of the likelihood ratio has an unusually larger order. A useful sufficient condition of the larger order is shown, and applied to neural networks. The order of the asymptotic likelihood ratio are shown in various cases of multilayer perceptrons. 1