Visualizing probabilistic models in Minkowski space: an analytical coordinate embedding
Han Kheng Teoh, Katherine N. Quinn, Jaron Kent-Dobias, Colin B. Clement, Qingyang Xu, James P. Sethna · arXiv (Cornell University) · 2019
We show that the predicted probability distributions for any $N$-parameter statistical model taking the form of an exponential family can be explicitly and analytically embedded isometrically in a $N+N$-dimensional Minkowski space. That is, the model predictions can be visualized as control parameters are varied, preserving the natural distance between probability distributions. All pairwise distances between model instances are given by the symmetrized Kullback-Liebler divergence. We give formulas for these isKL coordinate embeddings, and illustrate the resulting visualizations with the coin toss problem, the ideal gas, n sided die, the nonlinear least squares fit, and the Gaussian fit. We conclude by visualizing the prediction space of the two-dimensional Ising model, where we examine the manifold behavior near its critical point.