Retrain-free fully connected layer optimization using matrix factorization
Yi Jie Sun, Xuejiao Liu, Luhong Liang · 2017
The complexity of Deep Neural Networks (DNNs) hinders their implementation on embedded system with limited hardware resources. To deal with this issue, this paper presents a novel optimization algorithm based on semi-Nonnegative Matrix Factorization for Fully Connected layers (semi-NMF-based FC optimization). Compared with previous network surgery techniques, our proposed method optimizes network structure in a more implementation-friendly manner with full controllability and no requirement on retraining using training data. Simulations using AlexNet and VGG-16 on ImageNet task show the effectiveness and efficiency of semi-NMF-based FC optimization.