Expectation Propagation for Likelihoods Depending on an Inner Product of Two Multivariate Random Variables
Tomi Peltola, Pasi Jylänki, Aki Vehtari · 2014
We describe how a deterministic Gaussian posterior approximation can be constructed using expectation propagation (EP) for mod-els, where the likelihood function depends on an inner product of two multivariate random variables. The family of applicable models includes a wide variety of important linear latent variable models used in statistical ma-chine learning, such as principal component and factor analysis, their linear extensions, and errors-in-variables regression. The EP computations are facilitated by an integral transformation of the Dirac delta function, which allows transforming the multidimen-sional integrals over the two multivariate ran-dom variables into an analytically tractable form up to one-dimensional analytically in-tractable integrals that can be efficiently computed numerically. We study the result-ing posterior approximations in sparse prin-cipal component analysis with Gaussian and probit likelihoods. Comparisons to Gibbs sampling and variational inference are pre-sented. 1