A Lightweight Training-Free Method for Neural Architecture Search
Jhih-Cian Lin, Chun-Wei Tsai · 2024
The training-free score function was proposed in recent studies because it can be used to cut down the computation costs in evaluating a neural architecture compared to a complete training process. However, although most training-free score functions can dramatically accelerate the speed of a neural architecture evaluation, a certain number of misjudgments for neural architectures may still occur during the process of neural architecture search (NAS). To address this problem, this paper presents a new training-free score function and genetic algorithm to evaluate a neural architecture for NAS, called genetic algorithm for lightweight training-free neural architecture search (GALTNAS). The training-free score function proposed here will not only take into account the number of parameters and convolution layers of a neural architecture, but it also takes less computation costs than other training-free score functions. In order to evaluate the performance of the proposed algorithm, we compare it with other state-of-the-art training-free and non-training-free NAS algorithms. The experimental results show that GALTNAS outperforms all the other NASs in terms of the accuracy and computational cost for complex search spaces. The results also show that GALTNAS can provide a 10% improvement in accuracy compared to other non-weight-sharing, weight-sharing, and training-free methods.