Expectation propagation
Manfred Opper · 2015
Abstract Variational inference is a powerful concept that underlies many iterative approximation algorithms: expectation propagation, mean-field methods, belief propagation, and TAP equations can all be perceived in terms of this unifying framework. This chapter introduces the archetypal example of expectation propagation; after following its original derivation, we describe some of its properties, introduce some examples, and establish connections with the other approximation methods. The Gibbs free energy and its relation to these approximations, as well as double-loop algorithms for its minimization, are briefly discussed. Corrections by expansion about expectation propagation are then explained and, finally, some advanced inference topics and applications, such as recommender systems, Gaussian regression models and continuous-time stochastic dynamics, are explored.