An incremental learning approach for restricted boltzmann machines
Jongmin Yu, Jeonghwan Gwak, Sejeong Lee, Moongu Jeon · 2015
Determination of model complexity is a challenging issue to solve computer vision problems using restricted boltzmann machines (RBMs). Many algorithms for feature learning depend on cross-validation or empirical methods to optimize the number of features. In this work, we propose an learning algorithm to find the optimal model complexity for the RBMs by incrementing the hidden layer. The proposed algorithm is composed of two processes: 1) determining incrementation necessity of neurons and 2) computing the number of additional features for the increment. Specifically, the proposed algorithm uses a normalized reconstruction error in order to determine incrementation necessity and prevent unnecessary increment for the number of features during training. Our experimental results demonstrated that the proposed algorithm converges to the optimal number of features in a single layer RBMs. In the classification results, our model could outperform the non-incremental RBM.