Information Geometry and Statistical Pattern Recognition
Shinto Eguchi · 2004
This paper discusses a geometry associated with U-divergence including ideas of U-models, U-loss functions of two versions. On the basis of the geometry we observe that U-divergence projection of a data distribution p onto U-model MU associates with the Pythagorean relation for the triangle connection of p q and q∗, for any q of the U-model where q ∗ denotes the point of MU projected from p. This geometric consideration is implemented on the problem of sta-tistical pattern recognition. U-Boost algorithm proposed in the practical application is shown to pursue iteratively the U-divergence projection onto U-model evolving by one dimension according to one iteration. In particular U-Boost algorithm released from the probability con-straint reveals a novel property of statistical property beyond the notion of Fisher consistency, which helps us to understand the statistical meaning of AdaBoost. Key words AdaBoost, exponential family, logistic model, maximum likelihood, U-divergence,