Estimating Density Models with Truncation Boundaries
Song Liu, Takafumi Kanamori, Daniel J. Williams · arXiv (Cornell University) · 2019
Truncated densities are probability density functions defined on truncated domains. They share the same parametric form with their non-truncated counterparts up to a normalization term. Since the computation of their normalization term is usually infeasible, unnormalized models are used for parameter estimation. Score Matching is a powerful tool for fitting parameters in unnormalized models. However, it cannot be straightforwardly applied here as boundary conditions used to derive a tractable objective are usually not satisfied by truncated distributions. In this paper, we study parameter estimation for truncated probability densities using generalized SM. The choice of the weight function in generalized SM is critical to provide a computationally tractable and statistically preferable estimator even for complicated boundaries. As to the weight function, we use the distance function that is defined as the distance from a point in the domain to the boundary of the domain. We show the consistency of the proposed method as well as its link with the minimum Stein discrepancy estimator. The usefulness of our method is demonstrated by numerical experiments and real-world experiments.