Sparsifying to optimize over multiple information sources: an augmented Gaussian process based algorithm
Antonio Candelieri, Francesco Archetti · Structural and Multidisciplinary Optimization · 2021
Abstract Optimizing a black-box, expensive, and multi-extremal function, given multiple approximations, is a challenging task known as multi-information source optimization (MISO), where each source has a different cost and the level of approximation (akafidelity) of each source can change over the search space. While most of the current approachesfusethe Gaussian processes (GPs) modelling each source, we propose to use GPsparsificationto select only “reliable” function evaluations performed over all the sources. These selected evaluations are used to create an augmented Gaussian process (AGP), whose name is implied by the fact that the evaluations on the most expensive source areaugmentedwith the reliable evaluations over less expensive sources. A new acquisition function, based on confidence bound, is also proposed, including both cost of the next source to query and the location-dependent approximation of that source. This approximation is estimated through amodel discrepancymeasure and the prediction uncertainty of the GPs. MISO-AGP and the MISO-fused GP counterpart are compared on two test problems and hyperparameter optimization of a machine learning classifier on a large dataset.