Efficient Bayesian Hyperparameter Optimization Driven by Adaptive Trust Regions and Early Stopping
Shiyi Zhu, Shanshan Yu, Jing Jie · 2025
When training machine-learning models for brain–computer interface (BCI) decoding, practitioners often grapple with a large hyper-parameter space, where exhaustive tuning is hampered by heavy computational cost and noisy performance estimates. To address this challenge, we introduce a novel Bayesian-optimization framework that couples an adaptive trustregion strategy with a global reference early-stopping criterion. The algorithm dynamically expands or contracts its search radius based on consecutive improvements or failures, while using the best-seen validation curve as a stopping threshold to balance exploration with training cost.We evaluate the method on two representative BCI datasets–the Grid-Reaching Task (GRT) and the Center-Out Task (COT)–using two baseline decoders: a CNN-LSTM-Attention (CLA) network and a Temporal Convolutional Network (TCN). Systematic comparisons are made against Random Search (RS), conventional Bayesian Optimization (BO), Asynchronous Successive Halving (ASHA), Differential Evolution Hyper-band (DEHB), and our Trust-Region and Early-Stopping Bayesian Optimization (TRES-BO).Under a fixed 1-hour time budget, TRES-BO improves the mean correlation coefficient by 1.75 % over BO and 2.47 % over RS. With the number of trials held constant, TRES-BO reduces search time by 40–60 % relative to RS, achieving superior accuracy and efficiency. These results highlight TRES-BO as a practical, easily extensible paradigm for hyper-parameter optimization in BCI decoding and other compute-intensive applications.