Optimization of Stochastic Blackboxes with Adaptive Precision
Stéphane Alarie, Charles Audet, Pierre-Yves Bouchet, Sébastien Le Digabel · SIAM Journal on Optimization · 2021
In derivative-free and blackbox optimization, the objective function is often evaluated through the execution of a computer program seen as a blackbox. It can be stochastic, in the sense that an additive centered Gaussian random noise is present in the output. Sometimes, the distribution of the noise is tunable, and its standard deviation can be chosen at any execution of the blackbox. A common strategy to deal with such a situation is to define a sequence of standard deviation values monotonically decreasing to zero with the iterations to ensure convergence of algorithms because the noise is asymptotically dismantled. However, in practice a monotonic reduction of the standard deviation monotonically increases the computation time and makes the optimization process long. There is another strategy, which does not force the standard deviation per iteration to monotonically diminish. This work proposes an algorithmic framework adapted from the deterministic Mads algorithm, on which these two strategies can be expressed. Although these strategies are proved to be theoretically equivalent, tests on analytical problems and on an industrial blackbox with non-Gaussian noise distribution are presented to illustrate practical differences.