Improvements on style transfer from photographs to Chinese ink wash paintings based on generative adversarial networks
Tianyi Zhang · Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022) · 2023
Although voids play an important role in Chinese paintings, when generating Chinese ink wash paintings from real life photos of horses, background detection and void-leaving remain challenges in producing more credible paintings. To address the problem, a model with a two-stage framework is proposed in this paper. The framework divides the generation process into background lightening and style transformation. In the first stage, by training a pix2pix model with paired original photos and background-lightened photos, the pix2pix model is enabled to detect and lighten the background correctly in the style-transfer process, hinting where voids should be left. In the second stage, a Cycle-GAN model trained with unmatched photos and Chinese paintings achieves the style-transfer from pre-processed photos to Chinese ink wash paintings. In preparation for training the stage-Ⅰ model, the photos from a given dataset is processed in batches to get corresponding background-lightened photos. During the training of the stage-Ⅱ model, original photos instead of processed photos is used as comparatively data augmentation, making the model more robust and independent. Compared to the baseline model, the proposed model reaches a higher accuracy in both void-leaving and detail-enhancing in the experiments, resulting in more credible and delicate generated Chinese paintings.