Bias in optimization of environmentally adaptive control policies and a solution thereof
Ryo Ariizumi, Shuhei Sugiura, Toru Asai · 2017
Experiment-based black-box optimization is now under active research, because of its usefulness in a wide range of fields. Among related studies, some proposed an iterative algorithm for optimizing environmentally adaptive control policies using response surface method, which is expected to be useful for the applications in, e.g. mobile robots. A metric named unbiased expected improvement is the key to efficiently obtain the approximated optimal policy. This metric is intended to enhance the search in all possible environments, which is not possible if the standard expected improvement is used. This paper shows that there still exists a bias that should be attenuated to achieve efficient experiments and proposes a simple solution to get rid of this bias. Numerical tests show the effectiveness of our method.