Efficient Sample-based Neural Architecture Search with Learnable Predictor
Han Shi, Renjie Pi, Hang Xu, Zhenguo Li, James Tin-Yau Kwok, Tong Zhang · arXiv (Cornell University) · 2019
Neural Architecture Search (NAS) has shown great potentials in finding a better neural network design than human design. Sample-based NAS is the most fundamental method aiming at exploring the search space and evaluating the most promising architecture. However, few works have focused on improving the sampling efficiency for NAS algorithm. For balancing exploitation and exploration, we propose BONAS (Bayesian Optimized Neural Architecture Search), a sample-based NAS framework combined with Bayesian Optimization. The main components of BONAS are Sampler and Learnable Embedding Extractor. Specifically, we apply Evolution Algorithm method as our sampler and apply Graph Convolutional Network predictor as a surrogate model to adaptively discover and incorporate nodes structure to approximate the performance of the architecture. For NAS-oriented tasks, we also design a weighted loss focusing on architectures with high performance. Extensive experiments are conducted to verify the effectiveness of our method over many competing methods, e.g. 123.7x more efficient than Random Search and 7.5x more efficient than previous SOTA LaNAS for finding the best architecture on the largest NAS data set NAS-Bench-101.