Parallel tempering is efficient for learning restricted Boltzmann machines
Kyunghyun Cho, Tapani Raiko, Alexander A. Il'in · 2010
A new interest towards restricted Boltzmann machines (RBMs) has risen due to their usefulness in greedy learning of deep neural networks. While contrastive divergence learning has been considered an efficient way to learn an RBM, it has a drawback due to a biased approximation in the learning gradient. We propose to use an advanced Monte Carlo method called parallel tempering instead, and show experimentally that it works efficiently.