Semantic Image Synthesis with Trilateral Generative Adversarial Networks
Benyuan Li, Yue Yu, Menglan Wang · 2020
We present a new method that can synthesize photorealistic images from semantic label maps using GANs with a trilateral generator. Due to the limitation of network capacity, it is difficult for scene image generation networks to consider resolution and image fidelity at the same time. The solution is to use multi-path structure instead of the traditional single-path structure. In this work, we propose a novel trilateral Generative Adversarial Network (trilateral GAN), which has fewer parameters than other recent methods to synthesize 512*1024 images with high fidelity. Moreover, we improve the semantic consistency loss and feature matching loss by using the features before activation, which can make the synthetic images have sharper edges and richer textures. Finally, we replace all the Batch-Normalization (BN) layers with Spectral Normalization (SN) in the network and use Conditional Normalization Blocks (CNB) to avoid difficult convergence during training and make the synthesized images more realistic. The proposed network has a better performance and gets higher mIoU scores on COCO-Stuff, ADE20K and Cityscapes than the state-of-the-art methods.