MC-PDARTS: Multi-Cell Progressive Differentiable Architecture Search
Qun Liu, Yi Cai, Junying Chen · 2023
Recently, network architecture search has been practiced in more and more tasks, but its high cost makes it difficult for small research teams or individual researchers to follow. Differentiable architecture search (DARTS) algorithm provides an easier available solution in searching effective network architectures. Although DARTS algorithm has greatly reduced the search cost as compared with other evolution based or reinforcement learning based methods, there is still a long way to go before it is widely used, since gradient based methods consume a lot of GPU memory and its search space lacks diversity. Encouraged by the idea of progressive differentiable architecture search (P-DARTS), we propose a multi-cell progressive differentiable architecture search (MC-PDARTS) algorithm, which allows cells at different levels to learn different structures, and progressively searches the best structure for the corresponding level in each search stage. The proposed algorithm greatly enlarges the search space of the search network while reducing the memory consumption and search time in the search process, and alleviates the problem of performance collapse caused by too many skip connections which often occur in previous gradient based methods. Compared with the previous algorithms, the proposed MC-PDARTS algorithm achieves state-of-the-art performance on CIFAR-10 and CIFAR-100 datasets with around 4-hour search on a single Nvidia GTX1080Ti GPU. In order to reduce the performance loss in the transfer process, MC-PDARTS searches directly on a large target dataset (ImageNet) without proxy tasks, achieving state-of-the-art performance.