Fast Allocation of Gaussian Process Experts
Trung Nguyen, Edwin V. Bonilla · 2014
We propose a scalable nonparametric Bayesian regression model based on a mixture of Gaussian process (GP) experts and the inducing points for-malism underpinning sparse GP approximations. Each expert is augmented with a set of inducing points, and the allocation of data points to experts is defined probabilistically based on their prox-imity to the experts. This allocation mechanism enables a fast variational inference procedure for learning of the inducing inputs and hyperparam-eters of the experts. When using K experts, our method can run K2 times faster and use K2 times less memory than popular sparse methods such as the FITC approximation. Furthermore, it is easy to parallelize and handles non-stationarity straightforwardly. Our experiments show that on medium-sized datasets (of around 104 training points) it trains up to 5 times faster than FITC while achieving comparable accuracy. On a large dataset of 105 training points, our method sig-nificantly outperforms six competitive baselines while requiring only a few hours of training. 1.