AnimateAnywhere: Context-Controllable Human Video Generation with ID-Consistent One-shot Learning
Hengyuan Liu, Xiaodong Chen, Xinchen Liu, Xiaoyan Gu, Wu Liu · 2024
We demonstrate AnimateAnywhere, a personalized video generation framework that generates videos of a specific person with customized motions, scenes, and objects. Compared to existing approaches that animate a reference person image with a fixed background, AnimateAnywhere not only can preserve the consistency of the person but also can control the context like the scenes in the video. To achieve this goal, we first train a powerful base model using large-scale human images and videos with diverse scenes, poses, and captions to learn knowledge about contexts and human motions. Then, given a short video, we propose an ID-consistent one-shot learning method to obtain a personalized model by injecting the ID-related information into the pre-trained model. Finally, the user only needs to type in a text prompt to describe the expected scene/objects and select a reference motion, AnimateAnywhere can generate his/her video with the desired conditions.