Improved differentiable architecture search algorithm
Yuan Gao, Xiaohui Li, Changbao Li, Licai Wang, Jia He · 2023
As a gradient-guided search method, differentiable architecture search greatly reduces computational costs and improves search speed compared with traditional reinforcement learning methods and evolutionary methods that search for network structures in discrete spaces. However, when the number of search epochs is too large, the searched architecture will contain a lot of skip connections, resulting in a sharp decline in network performance. Aiming at this phenomenon, this paper designs a phased search process. With the deepening of the search stage, different early stopping rules are designed, so that the units located in different positions of the network can present different structures, effectively solving the performance crash problem caused by skip connections. At the same time, the design of edge normalization is introduced on the connection between nodes, and the difference between the weight parameters of different operations is enlarged by improving the loss function, which effectively improves the stability of the architecture search. The experimental results show that this method trades off a small amount of parameters and search time in exchange for an improvement in accuracy, and the verification accuracy on the cifar10 and cifar100 datasets has increased by 0.15% and 1.36%, respectively.