An Infinite Deep Boltzmann Machine
Xuan Peng, Xunzhang Gao, Xiang Li · 2018
The deep Boltzmann machine (DBM) is a powerful "deep" probabilistic model which learns a hierarchical representation of the data. However, choosing the size of each hidden layer of a DBM is difficult as the proper size of the model varies according to different tasks. Choosing a proper model size is a essential model selection problem for latent variable graphical models. This paper provides a new variant of DBM, called the infinite deep Boltzmann machine (iDBM), which can freely change the number of hidden units participating in the energy function of each layer. A greedy training method is proposed to pre-train our model, after which the size of each layer is fixed, and the model is transferred into an ordinary DBM. Experimental results on MNIST and CalTech101 Silhouettes indicate that iDBM can learn a generative and discriminative model as good as the original DBM, and has successfully eliminated the requirement of model selection for hidden layer sizes of DBMs.