Scalable and Robust Bayesian Inference via the Median Posterior
Stanislav Minsker, Sanvesh Srivastava, Lizhen Lin, David B. Dunson · 2014
Many Bayesian learning methods for massive data benefit from working with small subsets of observations. In particular, significant progress has been made in scalable Bayesian learning via stochastic approximation. However, Bayesian learning methods in distributed computing en-vironments are often problem- or distribution-specific and use ad hoc techniques. We pro-pose a novel general approach to Bayesian in-ference that is scalable and robust to corruption in the data. Our technique is based on the idea of splitting the data into several non-overlapping subgroups, evaluating the posterior distribution given each independent subgroup, and then com-bining the results. Our main contribution is the proposed aggregation step which is based on finding the geometric median of subset poste-rior distributions. Presented theoretical and nu-merical results confirm the advantages of our ap-proach. 1.