Differentiable Light-Weight Architecture Search
Yuxu Mao, Guoqiang Zhong, Yanan Wang, Zhaoyang Deng · 2021
In this paper, we propose a differentiable light-weight architecture search method, called DLWAS. It can search for CNNs with fewer parameters and floating point operations (FLOP-s) while maintaining the SOTA performance. Concretely, we first build a new light-weight search space, which contains the latest and effective light-weight operations, limiting the parameters and FLOPs from the source of neural architecture search (NAS). Secondly, we propose an effective neural architecture optimization method, which results in a more sparse and robust topology for differentiable NAS. Experimental results show that DLWAS achieves an error rate of 2.76% on CIFAR10 comparable to the state-of-the-art methods, using only 2.4M params and 336M FLOPs. The parameters and FLOPs are reduced by 30% and 36% compared with the closest counterpart DARTS, respectively. On ImageNet, our model achieves 3.2% better top-1 accuracy than the SOTA MobileNet, while using fewer parameters and FLOPs.