Multimodal Image-to-Image Translation by Enforcing Bi-Cycle Consistency
Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor J. Darrell, Oliver Wang, Eli Shechtman, Alexei A. Efros · Neural Information Processing Systems · 2017
Many image-to-image translation problems are ambiguous, with a single input image corresponding to multiple possible outputs. In this work, we aim to model a distribution of possible outputs in a conditional generative modeling setting. The ambiguity of the mapping is encoded in a low-dimensional latent vector, which can be randomly sampled at test time. A generator learns to map the input, along with the latent code, to an output. We explicitly enforce cycle consistency between the latent code and the output. Encouraging invertibility helps prevent a many-to-one mapping from the latent code to the output during training, also known as the problem of mode collapse, and helps produce more diverse results. We evaluate the relationship between perceptual realism and diversity of images generated by our method, and test on a variety of domains.