An Efficient Neural Network Design for Image Super-Resolution with Knowledge Distillation
Tuan Nghia Nguyen, Xuan Truong Nguyen, Kyujoong Lee, Hyuk‐Jae Lee · 2023
This paper proposes a new neural network design for efficient image super-resolution inference. Employing complex-simple sub-networks, the proposed design samples feature to dynamically choose an execution path, leading to considerable computation reduction. However, uniformly random sampling generally causes a large accuracy drop due to highly different feature maps obtained by the sub-networks. To address the problem, we propose two simple yet effective techniques considering both the training and inference stages. First, Knowledge Distillation is utilized during training to minimize the feature map difference. Second, a gradient image which is obtained with the well-known Sobel filter guides the sampling by assigning points on edge and texture regions to the complex sub-network. The experimental results show that the proposed design reduces 50% of computations when only 20% of feature maps are computed by the complex sub-network. More importantly, the proposed sampling method enhances the restoration accuracy by 0.3 dB on average compared to the uniformly random sampling method.