Bayesian inference for multi-level non-stationary Gaussian processes
Karla Monterrubio Gómez · Warwick Research Archive Portal (University of Warwick) · 2019
The complexity of most real-world phenomena requires the use of flexible models that capture intricated features present in the data. Gaussian processes (GPs) have proven valuable tools for this purpose due to their non parametric and probabilistic nature. Nevertheless, the default approach when modelling with GPs is to assume stationarity. This assumption permits easier inference but can be restrictive when the correlation of the process is not constant across the input space. This thesis investigates a class of non-stationary priors that enhance flexibility while retaining interpretability. These priors assemble GPs through input-varying parameters in the covariance. Such hierarchical constructions result in high-dimensional correlated posteriors, where Bayesian inference becomes challenging and notably expensive due to the characteristic computational constrains of GPs. Altogether, this thesis provides novel approaches for scalable Bayesian inference in 2-level GP regression models. First, we use a sparse representation of the inverse non-stationary covariance to develop and compare three different Markov chain Monte Carlo (MCMC) samplers for two hyperpriors. To maintain scalability when extending the approach to multi-dimensional problems, we propose a non-stationary additive Gaussian process (AGP) model. The efficiency and accuracy of the methodology are demonstrated in simulated experiments and a computer emulation problem. Second, we derive a hybrid variational-MCMC approach that combines low-dimensional variational distributions with MCMC to avoid further distributional and independence restrictions on the posterior of interest. The resulting approximate posterior includes an intractable likelihood that when approximated with a small-order Gauss-Hermite quadrature results in poor predictive performance. In this case, an extension to higher-dimensional settings requires specific assumptions of the non-stationary covariance. Lastly, we propose a pseudo-marginal algorithm that uses a block-Poisson estimator to circumvent numerical integration in the variationally sparse model. This strategy demonstrates an improvement in predictive performance, can be computationally more efficient, and is generally applicable to other GP-based models with intractable likelihoods.