Double Loss Block Neural Architecture Search
Yang Liu, Jun Lu · 2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC) · 2022
Neural architecture search aims to automatically design network architectures by machines, which is expected to bring about a new revolution in machine learning. Despite high expectations, the effectiveness and efficiency of existing neural architecture search solutions are unclear, and the inefficiency of neural architecture search solutions is mainly attributed to inaccurate architecture evaluation. In this work, this paper divides the large search space of neural architecture search into multiple modules, effectively reducing the error caused by shared parameters to ensure that the candidate architectures in the modules are fully trained. By adopting a block search method, this paper can Evaluate all candidate architectures in a module, addressing the problem of inaccurate evaluation. In addition, high-quality data is necessary. Under the guidance of the teacher model, combined with the ground truth, the neural architecture search is supervised by the double loss function. Under the constraint of the same amount of computation, the method in this paper achieves 77.3% top-level accuracy on ImageNet., which is 1.0% higher than the efficient network EfficientNet-B0, and 0.3% higher than the DNA method without additional computation. Therefore, the double loss block neural architecture search model proposed in this paper significantly improves the effectiveness of neural architecture search.