A supervisory hierarchical control approach for text to 2D scene generation
Yu Cheng, Zhiyong Sun, Sheng Bi, Congjian Li, Ning Xi · 2017
Photo-realistic image generation conditioned on text is an important problem and would have a wide range of applications. The gap between the high level abstract semantic information and the detailed low level image elements and operations on them hampers utilization of text-to-image systems. Many research works have been proposed to bridge the gap. However, most existing approaches do not allow operations on objects during the image synthesis. To generate photo-realistic images with more flexibility, in this work, we propose a two level hierarchical control system framework for image synthesis. The high level system acts as task planner and cope with abstract concepts extracted from input text. While the lower level system plays the role of action planner, which takes command from task planner and generates an action sequence for lower level implementation. The proposed approach allows control of the location and dimension of each object appeared in the text input, which helps to generate more complex scene. In addition, the proposed method allows to synthesize images with higher resolution. Experimental results have been provided to validate the proposed method.