Efficient Constrained Gaussian Process Approximation Using Elliptical Slice Sampling
Hassan Maatouk, Didier Rullière, Xavier Bay · Bayesian Analysis · 2025
In this paper, Bayesian shape-restricted function estimation using constrained Gaussian processes (GPs) is revisited. The GP approximation proposed by Maatouk and Bay (2017) is considered, which satisfies a wide range of shape constraints, such as monotonicity, convexity, and boundedness constraints across the entire domain. To generate samples from the resulting posterior distribution, we employ a recently efficient circulant embedding technique. This technique involves incorporating a smooth relaxation of the constraint set into the likelihood, a prior distribution, and an efficient Markov chain Monte Carlo (MCMC) sampler. Our contribution in this article is fourfold. First, we extend this approach to address sets of general linear inequalities, enabling the incorporation of multiple and complex shape constraints. This generalization allows the proposed methodology to be easily adapted to other Bayesian linear models. We derive an efficient formula for the log-likelihood function that significantly reduces computational complexity and improves runtime efficiency in high-dimensional settings. Second, we update the approximation parameter of the set of linear constraints at each MCMC iteration. This ensures the stability and the convergence of the proposed MCMC sampler. Third, we explore efficient samplers for generating posterior and prior distributions, including Hamiltonian Monte Carlo and the Fast Fourier Transform. Additionally, we adopt a large-scale, highly efficient approach for prior sampling, yielding significant computational advantages. Fourth, we investigate the capability of this approach to handle higher-dimensional input spaces and manage a large number of observations. The proposed approach demonstrates excellent flexibility and accuracy in both synthetic and real-world data studies.