Accelerating sparse restricted Boltzmann machine training using non-Gaussianity measures
Sander Dieleman, Benjamin Schrauwen · Ghent University Academic Bibliography (Ghent University) · 2012
In recent years, sparse restricted Boltzmann machines have gained popularity as unsupervised feature extractors. Starting from the observation that their training process is biphasic, we investigate how it can be accelerated: by determining when it can be stopped based on the non-Gaussianity of the distribution of the model parameters, and by increasing the learning rate when the learnt filters have locked on to their preferred configurations. We evaluated our approach on the CIFAR-10, NORB and GTZAN datasets. 1