EDNAS: An Efficient Neural Architecture Design based on Distribution Estimation
Zhenyao Zhao, Guang-en Zhang, Min Jiang, Liang Feng, Kay Chen Tan · 2020
Neural architecture search (NAS) is the process of automatically searching for the best performing neural model on a given task. Designing a neural model requires a lot of time for experts, NAS's automated process effectively solves this problem and makes neural networks easier to promote. Although NAS has achieved excellent performance, its search process is still very time consuming. In this paper, we propose a neural architecture design method based on distribution estimation method called EDNAS, a fast and economical solution to design neural architecture automatically. In EDNAS, we assume that the best performing architecture obeys a certain probability distribution in search space. Therefore, NAS can be transformed to learning this probability distribution. We construct a probability model on the search space, and search for this probability distribution by iterating the probability model. Finally, an architecture that maximizes the performance on a validation set is generated from this probability distribution. Experiment shows the efficiency of our method. On CIFAR-10 dataset, EDNAS discovers a novel architecture in just 4 hours with 2.89% test error, which shows efficent and strong performance.