Maximum reconstruction probability training of Restricted Boltzmann machines with auxiliary function approach
Norihiro Takamune, Hirokazu Kameoka · 2014
Restricted Boltzmann machines (RBMs) are stochastic neural networks that can be used to learn features from raw data. They have attracted particular attention recently after being proposed as building blocks for deep belief network (DBN) and have been applied with notable success in a range of problems including speech recognition and object recognition. The success of these models raises the issue of how best to train them. At present, the most popular training algorithm for RBMs is the Contrastive Divergence (CD) learning algorithm. We propose deriving a new training algorithm based on an auxiliary function approach for RBMs using the reconstruction probability of observations as the optimization criterion. Through an experiment on parameter training of an RBM, we confirmed that the present algorithm outperformed the CD algorithm in terms of the convergence speed and the reconstruction error when used as an autoencoder.