Realizing GANs via a Tunable Loss Function
Gowtham R. Kurri, Tyler Sypherd, Lalitha Sankar · 2021
We introduce a tunable GAN, called $\alpha$-GAN, parameterized by $\alpha\in$(0, $\infty$], which interpolates between various f-GANs and Integral Probability Metric based GANs (under constrained discriminator set). We construct $\alpha-$ GAN using a supervised loss function, namely, $\alpha-$ loss, which is a tunable loss function capturing several canonical losses. We show that $\alpha-$ GAN is intimately related to the Arimoto divergence, which was first proposed by Österriecher (1996), and later studied by Liese and Vajda (2006). We posit that the holistic understanding that $\alpha-$ GAN introduces will have practical benefits of addressing both the issues of vanishing gradients and mode collapses.