Enhancing Bayesian Optimization with NLML-Based Transfer Learning
Yanchen Li, Taku Tsuzuki · 2024
Bayesian optimization (BO) is a powerful machine learning method used to solve black-box optimization problems, such as exploring optimal conditions for cell culturing. For problems with high evaluation costs, transfer learning from similar source tasks helps BO find solutions more quickly, thereby reducing costs. A popular method of transferring learning is to merge the surrogate models of previous tasks through linear combination. To calculate the weights for this linear combination in a fine-grained manner, we propose adaptive distributions combination (ADC), a transfer learning method that optimizes the weights using negative log marginal likelihood (NLML). NLML directly optimizes the merged surrogate model to fit the distribution of the target function, thereby building an accurate surrogate model. Our experimental results indicate that ADC helps BO explore better solutions than ranking-based methods.