Supernets Based Image Classification
S. Sethu Selvi, Anirudh Shishir, Bhagawankripa Mehta, Jasdeep Singh Bedi, Chethan B A · 2023
Designing of Deep Neural Networks (DNN) is accelerated through an efficient approach known as Neural Architecture Search (NAS). A search space with various neural network architectures for performing a deep learning task is sampled through this approach. In practice, designing of neural networks is automated by NAS approach which drastically reduces the training time as compared to hand design. But critical problems tend to arise as the datasets become larger. The complexity in the hidden layers and computational cost is increased due to which vanishing gradient comes into picture. In this paper, an optimized approach based on NAS algorithms is proposed. Neural architectures in the search space are combined to form a single model called the SuperNet. A candidate architecture also known as the ensemble, is generated from the SuperNet by training the SuperNet once and sampling it. A wide range of parameters are evaluated with the validation set for the ensemble. The proposed algorithm is compared with existing NAS approaches and a conclusion is drawn that the SuperNet approach for NAS is effective in automating the neural network design. For CIFAR-10 dataset, the proposed algorithm provides 93.40% accuracy with GPU time of 8.78 hours compared to 75.82% accuracy with 17 days computation for conventional NAS algorithm.