Approximate Inference
Shuang Wang, Yong Fang, Samuel Cheng · 2017
Approximate InferenceAs shown in the previous chapter, the sum-product algorithm is a powerful probabilistic inference technique for efficiently computing posterior probabilities over discrete variables of small alphabet sizes or continuous variables of linear Gaussian distributions.However, it cannot handle a discrete variable with a medium alphabet size as the computational complexities increase exponentially with the alphabet size.Moreover, for continuous variables with nonlinear Gaussian distribution, the sum-product is also infeasible as the integration may not have a closed-form solution.To tackle these difficulties, two common workarounds in approximate inference are either to discretize the variable through sampling techniques or to parametrize the variables through variational inference, where the sampling and the variational methods are also known as stochastic and deterministic approximation schemes.In this chapter we will focus on both the stochastic approximation and the deterministic approximation.