Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks
Evan Becker, Parthe Pandit, Sundeep Rangan, Alyson K. Fletcher · 2023
Generative Adversarial Networks (GANs) are a popular formulation to train generative models for complex high dimensional data. The standard method for training GANs involves a gradient descent-ascent (GDA) procedure on a minimax optimization problem. This procedure is hard to analyze in general due to the nonlinear nature of the dynamics. We study the local dynamics of GDA for training a GAN with a kernel-based discriminator. This convergence analysis is based on a linearization of a nonlinear dynamical system that describes the GDA iterations, under an isolated points model assumption from [2]. Our analysis brings out the effect of the learning rates, regularization, and the bandwidth of the kernel discriminator, on the local convergence rate of GDA. Importantly, we show phase transitions that indicate when the system converges, oscillates, or diverges. We also provide numerical simulations that verify our claims. A full version with complete proofs is available on arXiv [3].