Gauge Optimization via ADMM for Approximate Inference
Sungsoo Ahn, Michael Chertkov, Jinwoo Shin · arXiv (Cornell University) · 2017
Computing partition function is the most important inference task arising in applications of Graphical Models (GM). Since it is computationally intractable, approximate algorithms are used to tackle the problem. In this paper, by relying on the technique, coined gauge transformation, modifying GM factors such that the partition function stay the same (invariant), we propose two optimization formulations which generalize the Bethe Free Energy, Belief Propagation approach. Then, we show that the optimizations can be solved efficiently by Alternating Direction Method of Multipliers (ADMM) algorithms. Our first algorithm provides deterministic lower bounds of the partition function. The algorithm is exact for GMs over a single loop with a special structure, even though the popular Belief Propagation algorithm performs badly in this case. Our second algorithm is of a randomized, Monte Carlo, type. It lowers sample variance, which can be further reduced with the help of annealed/sequential/adaptive importance sampling. The experiments show that the newly proposed Gauge-ADMM algorithms outperform other known algorithms for the approximate inference task.