A Fast GP Regression Method Using Banded Sparsification of Inverse Covariance
Yufeng Shi, Linfeng Xu, Jianliang Lai, Yifan Ju · 2023
Modeling and prediction using Gaussian processes (GPs) have widespread applications in machine learning, information processing, automatic control, biology, and other fields. However, GP regression requires a computational complexity of $O(N^{3})$. To overcome this issue, this paper proposes a fast GP regression method based on a banded sparsification of the inverse covariance matrix. It follows that we first propose an approximate inversion algorithm rooted in the Kullback-Leibler (KL) divergence, which exhibits approximately linear computational complexity. Furthermore, we demonstrate it is essentially a higher-order Gaussian Markov process (GMP) model and minimum mean square error (MMSE) estimation. Next, by transforming the inverse covariance matrix into a banded sparse structure, we derive a more efficient GP regression algorithm, of which the core is to employ a high-order GMP model to effectively approximate the general GP.