GroupifyVAE: from Group-based Definition to VAE-based Unsupervised Representation Disentanglement.
Tao Yang, Xuanchi Ren, Yuwang Wang, Wenjun Zeng, Nanning Zheng, Pengju Ren · arXiv (Cornell University) · 2021
The key idea of state-of-the-art VAE-based unsupervised representation disentanglement methods is to minimize total correlation of latent variable distributions. However, it has been proved that VAE-based unsupervised disentanglement can not be achieved without introducing other inductive bias. In this paper, we address VAE-based unsupervised disentanglement by leveraging constraints derived from Group Theory based definition as non-probabilistic inductive bias. More specifically, inspired by nth dihedral group (the permutation group for regular polygons), we propose a specific form of definition and prove its two equivalent conditions: isomorphism and the constancy of permutations. We further provide an implementation of isomorphism based on two Group constraints: Abel constraint for exchangeability and Order constraint for cyclicity. We then convert them into a self-supervised training loss that can be incorporated into VAE-based models to bridge their gaps from Group Theory based definition. We train 1800 models covering most prominent VAE-based models on five datasets to verify effectiveness of our method. Compared to original models, Groupidied VAEs consistently achieve better mean performance with smaller variances, and make meaningful dimensions controllable.