Stepwise PathNet: Transfer Learning Algorithm to Improve Network Structure Versatility
Shunsuke Imai, Hajime Nobuhara · 2018
Transfer learning can train a neural network for a target task's small dataset using a source task's pre-trained network; however, catastrophic forgetting, where the knowledge of the pre-trained network disappears during transfer learning, is problematic in this setting. PathNet was proposed to address this problem. However, PathNet can only be applied to modular neural network cases:thus, a non-modular pre-trained neural network is unavailable. Consequently, PathNet cannot be used to improve network structure versatility. Therefore, we propose Stepwise PathNet to improve versatility by considering the layers of a non-modular pre-trained neural network as modules. The performances of the proposed Stepwise and original PathNet methods were compared using the CIFAR-10 dataset (10 classes, and 60,000 images), and the results confirm the proposed method's potential to stabilize learning curves and accelerate learning to 45%.