A Feedback-inspired Super-network Shrinking Framework for Flexible Neural Architecture Search
Xuan Rao, Bo Zhao, Derong Liu, Haowei Lin · 2023
Neural architecture search (NAS) has been proved to be effective in automatically designing the architectures of deep neural networks (DNNs). Currently, most of cell-based NAS methods are subjected to the prior rules of human experts on neural architectures (e.g., the topology-sharing strategy), but it is not in line with the sustainable development of automated machine learning (Auto-ML). In this paper, a feedback-inspired super-network shrinking (FSS) method is developed under the framework of differentiable architecture search (DARTS) such that neural networks with flexible architectures are derived without strong reliance on human experts. In FSS, the information entropy is utilized to measure the sparsity of the super-network and an adaptive shrinking method is developed to balance the cross-entropy and the sparsity entropy. The experiments on CIFAR10/100 demonstrate that the proposed ESF derives a set of neural architectures with competitive performance-computation trade-offs in one single search procedure, which is superior to the state-of-the-art NAS methods.