A Priori Independence for Deep Generative Models
Rastin Rastgoufard, AbdulRahman Alsamman · 2018
Deep Generative Models are powerful learning tools that utilize an autoencoder structure for generating data and inferring latent variables. The most basic version has a single latent variable that encodes the training data in an unsupervised manner, and the extension to semi-supervised learning combines a latent class label with a continuous latent variable whose purpose is to provide the variations in the data that are not caused by the class label. The structure of the two-variable generative model should imply that both the class label and the continuous variable are free parameters that can be chosen to generate data, that is, the two are a priori independent. However, we show that this is the case only for certain data sets. We propose two objective functions for guiding the variables to be a priori independent, we use a novel training procedure to optimize the objectives, and we show experimentally that the objectives successfully produce the desired independence. We perform all of our experiments on the multi-labeled NORB dataset.