Restricted Boltzmann Machines: an Eigencentrality-based Approach
Andrew A. Skabar · 2019
Restricted Boltzmann Machines (RBMs) are usually trained using contrastive divergence learning, which requires setting suitable values for a number of hyper-parameters, and requires a degree of practical know-how. This paper presents a radically different approach to RBMs, which we refer to as the EigenCentrality-based Restricted Boltzmann Machine (ECRBM). Rather than actually `training' the RBM, the dataset is first represented as a bipartite graph consisting of a layer of attribute-value nodes and a layer of object nodes, with the representation chosen such that the components of an eigenvector calculated from the graph can be used to estimate the conditional probabilities of the attribute values. Matrix factorization is then used to replace this graph with a smaller graph, having the important property that eigenvectors calculated on this smaller graph preserve the interpretation of the eigenvector components as conditional probabilities. The model can be used to estimate and sample from any conditional or marginal distribution, and can therefore be applied to a diverse range of machine learning tasks including classification, regression, missing value imputation, outlier detection, and random vector generation. It can be applied to mixed-attribute datasets as easily as it can be applied to datasets containing variables of the same modality. We demonstrate the method by applying it to classification and random vector generation on a number of mixed-attribute datasets.