NAS-SW: Efficient Neural Architecture Search with Stage-Wise Search Strategy
Jikun Nie, Yeming Yang, Qingling Zhu, Qiuzhen Lin · 2024
A number of Neural Architecture Search (NAS) methods have been proposed to automate the design of Convolutional Neural Networks (CNNs). However, they typically require significant computational resources and exhibit search instability. To save computational costs, many NAS methods design only a single group of cell architectures (i.e., a small part of the total network architecture), which are simply stacked to form the final network. This leads to a lack of diversity in network architecture design, thereby reducing network performance. To alleviate this problem, we propose a Particle Swarm Optimization (PSO)-based Stage-Wise NAS algorithm (named NAS-SW). Initially, we introduce a stage-wise architecture search strategy, which can find different cell architectures at various search stages. Then, we propose a grouping architecture update strategy, which can realize effective information transfer between cell architectures rapidly. Importantly, these strategies do not add extra computational costs. Experiments demonstrate that NAS-SW has superior performance and better efficiency. NAS-SW only needs 0.37 GPU days to design a CNN on CIFAR-10. Additionally, on the CIFAR-10 and CIFAR-100 datasets, the network architectures searched by NAS-SW achieve top-1 validation accuracies of 97.55% and 83.78%, respectively.