Does Identity Mapping Really Help in ResNet?
Yoo-Kyung Lee, Dong-Hwan Lee, Jae-Hun Choi, Jang‐Hee Yoo, Seung‐Ik Lee · 2021 International Conference on Information and Communication Technology Convergence (ICTC) · 2021
ResNet proposed residual learning using skip connections that perform identity mapping. However, as shown in the study [1], identity mappings for skip connections may not be the optimal or at least not the only way to get a good performance in ResNet. In this paper, we explore various types of skip connections, including shortcuts with or without identity mapping, to see their effects on the performance of ResNet and provide a comparison of the various methods. Our extensive experiments suggest that the use of identity shortcuts is the most appropriate than other methods in terms of model simplicity and performance.