Vanilla Probabilistic Autoencoder
Sebastian Ciobanu · EPiC series in computing · 2021
The autoencoder, a well-known neural network model, is usually fitted using a mean squared error loss or a cross-entropy loss. Both losses have a probabilistic interpretation: they are equivalent to maximizing the likelihood of the dataset when one uses a normal distribution or a categorical distribution respectively. We trained autoencoders on image datasets using different distributions and noticed the differences from the initial autoen- coder: if a mixture of distributions is used the quality of the reconstructed images may increase and the dataset can be augmented; one can often visualize the reconstructed im- age along with the variances corresponding to each pixel. The code which implements this method can be found at https://github.com/aciobanusebi/vanilla-probabilistic-ae.