Model-based Design for GPs
Robert B. Gramacy · 2020
Model-based alternatives all assume something, like a Gaussian process (GP) kernel hyperparameterization, which is where the risk comes from. A model-based design is one where the model says what X n it wants according to a criterion targeting some aspect of its fit. Example targets include the quality of estimates for parameters or hyperparameters, or accuracy of predictions at particular inputs out-of-sample, or over the entire input space. In general, model-based optimality comes at potentially substantial computational cost. The simplest sequential design scheme for GPs, but it’s of course more widely applicable, involves choosing the next point to maximize predictive variance. Observe dense coverage along the boundary, a telltale sign of maxent design. The degree of space-fillingness could be improved, but by proceeding sequentially less faith is required in the quality of an initial hyperparameterization.