Exponential manifold by reproducing kernel Hilbert spaces
Kenji Fukumizu · Cambridge University Press eBooks · 2009
The purpose of this chapter is to propose a method of constructing exponential families of Hilbert manifolds, on which estimation theory can be built. Although there have been works on infinite-dimensional exponential families of Banach manifolds (Pistone and Sempi 1995, Gibilisco and Pistone 1998, Pistone and Rogantin 1999), they are not appropriate for discussing statistical estimation with a finite sample; the likelihood function with a finite sample is not realised as a continuous function on the manifold. The proposed exponential manifold uses a reproducing kernel Hilbert space (RKHS) as a functional space in the construction. A RKHS is defined as a Hilbert space of functions such that evaluation of a function at an arbitrary point is a continuous functional on the Hilbert space. Since evaluation of the likelihood function is necessary for the estimation theory, it is very natural to use a manifold associated with a RKHS. Such a manifold can be either finite or infinite dimensional depending of the choice of RKHS. This chapter focuses on the maximum likelihood estimation (MLE) with the exponential manifold associated with a RKHS. As in many non-parametric estimation methods, straightforward extension of MLE to an infinite-dimensional exponential manifold can be an ill-posed problem; the estimator is chosen from the infinitedimensional space, while only a finite number of constraints is given by the sample. To solve this problem, a pseudo-maximum likelihood method is proposed by restricting the infinite-dimensional manifold to a series of finite-dimensional sub-manifolds, which enlarge as the sample size increases. Some asymptotic results in the limit of infinite sample are shown, including the consistency of the pseudo-MLE.