Learning Depth Information in Layout for Sketch Generation from Scene Graph
Yuming Sun, Hangyu Lin, Chen Liu, Yanwei Fu · 2021
The research of generating images from scene graphs has become a hot topic, benefiting from the success of Generative Adversarial Network (GAN). Previous works in this field are still challenged by the complexity of texture pattern and object structure in images. In fact, it is more desirable to generate sketches directly, rather than image synthesis from scene graphs. The sketch is an abstract and iconic image representation, which describes the object structure and scene layout well but ignoring complex texture patterns, leading to a more reasonable generation task. Furthermore, there are two real-world tasks can illustrate the importance of this problem: Courtroom sketch and Crime Scene sketch, which are less studied before. To this end, we, for the first time, study the task of generating sketches from scene graphs. Essentially, the main novelties are two folds. First, a new sketch generation framework is developed which is trained with both newly designed perceptual loss and adversarial loss. Second, we propose a new layout encoding block to learn the depth information for each object. Extensive experiments on several widely used datasets validate the good performance of the proposed approach.