Search Space Reconstructing for Neural Architecture Search
Yuzhuo Gao, Hao Li, Chao Wu · 2024
Neural architecture search (NAS) has shown its great potential in finding outstanding network architecture. For NAS, a predefined search space is required to specify the searching principle for the search strategy. Search space defines the structural paradigm that searching methods can explore, directly predetermining the best network architecture search method can find. It is a vital but challenging problem to construct a great search space. Generally, a good search space is expected to exclude human bias and be large enough to cover a wider variety of model architectures. Existing NAS works manually design their search space according to prior knowledge and the scale of search space heavily suffers from the limitation of computational cost for it can grow exponentially when a new operation is added to search space. To maintain the advantage of a large search space while meeting the limitations of computational cost, we propose search space reconstruction (SSR), a simple but effective approach to construct search space based on dataset and task. Experiments demonstrate that a reconstructed search space can dramatically reduce time consumption during the searching phase and enhance the performance of search results. Specifically, we apply a new search space constructed by SSR to DARTS, without any other modification, achieving a 0.68% error rate promotion immediately on CIFAR-10.