Estimation of unnormalized statistical models without numerical integration
Michael U. Gutmann, Aapo Hyvärinen · 2013
Parametric statistical models of continuous or discrete val-ued data are often not properly normalized, that is, they do not integrate or sum to unity. The normalization is essen-tial for maximum likelihood estimation. While in princi-ple, models can always be normalized by dividing them by their integral or sum (their partition function), this can in practice be extremely difficult. We have been developing methods for the estimation of unnormalized models which do not approximate the partition function using numeri-cal integration. We review these methods, score matching and noise-contrastive estimation, point out extensions and connections both between them and methods by other au-thors, and discuss their pros and cons. 1.