Scaling Gaussian Processes
Yanshuai Cao · TSpace (University of Toronto) · 2018
We explore ways to scale Gaussian processes (GP) to large datasets. Two methods with different theoretical and practical motivations are proposed. The first method solves the open problem of efficient discrete inducing set selection in the context of inducing point based approximation to full GPs. When inducing points need to be chosen from the training set, the proposed method is the only principled approach to date for joint tuning of inducing set and GP hyperparameters while scaling linearly in the number of training set size during learning. Empirically it achieves a trade-off between speed and accuracy that is comparable to other state-of-arts inducing point GP methods. The second method is a novel framework for building flexible probabilistic prediction models based on GPs that is simple to parallelize and highly scalable. Referred to as transductive fusion, this second approach learns separate GP experts whose predictions are combined in ways that depend on test point locations. A number of new models are proposed in this new framework. Learning and inference in these new models are straightforwardly parallel, and predictive accuracy is shown to be satisfactory empirically.