Efficient Multi-Step Lookahead Bayesian Optimization with Local Search Constraints
Joel A. Paulson, Farshud Sorouifar, Ankush Chakrabarty · 2022 IEEE 61st Conference on Decision and Control (CDC) · 2022
Bayesian optimization (BO) is a data-efficient approach for optimizing expensive-to-evaluate black-box functions that suffer from noisy evaluations. Traditional BO algorithms ignore the relationship between consecutive input values, which is known to lead to significant "jumps" in the search space that cannot be implemented in practice, especially in online experimental systems. For example, in performance-driven control applications, large changes in the chosen setpoint parameters may trigger fail-safe mechanisms or lead to violation of critical safety constraints. In such applications, it is necessary to limit the allowable search space at each BO iteration, which can be done by incorporating local search constraints into the original problem setting. In this paper, we show how this novel BO setting can be cast as a Markov decision process (MDP) for which the optimal policy is characterized by a dynamic programming (DP) problem. This DP problem generally being computationally prohibitive to solve, we adopt approximate DP methods such as rollout with fast policy search to derive an efficient multi-step lookahead BO policy. We also propose a novel base policy needed for the rollout algorithm, which explicitly incorporates the local search restrictions in an intuitive manner. Lastly, we show via numerical experiments that our proposed multi-step lookahead BO policy outperforms existing methods on a benchmark non-convex global optimization problem.