Bridging Script and Animation Utilizing a New Automatic Cinematography Model
Zixiao Yu, Enhao Guo, Haohong Wang, Jian Ren · 2022
For animated film-making, automatic cinematography is an effective approach for junior filmmakers to speed up the process. The virtual camera placement in the 3D environment can be automatically conducted by auto-cinematography algorithms. In the literature, the algorithms proposed were mainly optimized from aesthetic point of view, thus the scene captured by virtual cameras may not effectively represent the intention of the script sometimes. In this paper, a new model that incorporates the fidelity and aesthetic models in a unified framework was proposed, so that the visual presentation of the input script and the compliance of the generated video with given cinematography specifications can be jointly considered during the optimization process. In this way, the problem of virtual camera placement is translated into an optimization problem that can be solved through dynamic programming that optimizes the computational efficiency. Experimental results shows that the proposed algorithm has quality improvement up to 35% compared to the earlier methodology and the content of the video is easier to understand. More video examples can be reviewed https://youtu.be/0PUdV6OeMac.