Variational Autoencoder for Data Analytics in Internet of Things Based on Transfer Entropy
Stephen D. Liang · IEEE Internet of Things Journal · 2021
Variational autoencoders (VAEs) are generative models which combine deep learning and Bayesian machine learning. The VAEs are trained via minimizing the loss function, and the most popular loss function in VAEs is the evidence lower bound function in which the Kullback–Leibler divergence has been used. Motivated by the idea that information flow from VAEs input to output should be reflected in the loss function, we propose to incorporate transfer entropy (TE) to loss function to quantify the information flow. Subsequently, we apply our TE-based VAEs to data analytics in Internet of Things, including data compression and generative modeling. Simulation results show that our TE-based VAEs works much better than the Kullback–Leibler divergence-based VAE in terms of reconstruction error and label error. We further analyze the latent space to clarify why our TE-based VAE performs better. We compare our TE-based VAE against compressive sensing and demonstrate that our TE-based VAE could have 30 times stronger compression power than compressive sensing.