Single Stage Multi-pose Virtual Try-on Network with Deformable Attention
Siyu Liu, Peiyan Lu, Zizhao Wu · 2023
Current methods for image-based virtual try-on are limited to a single-pose scenario, where they generate an image of a source people wearing a given garment. However, this limitation restricts their practical applicability, as they cannot generate try-on images for different poses of the person. To overcome this limitation, we present MDAVTON, a single stage multi-pose virtual try-on network with deformable attention. Our framework incorporates two key designs. Firstly, we introduce a flow estimation module that calculates the garment’s flow and the person’s flow using deformable attention machine. Specifically, the flow estimation module computes the two-dimensional flow fields by sampling from a fixed number of key points and estimating the weights of the sampled points. These sparse attentional weights are applied to the corresponding sampled points so that the sampled points are gradually biased to the points with the strongest correspondence to the source feature. Secondly, we design a conditional generation module to refine the generation results at the pixel level. This module ensures that the generated images are both realistic and capable of retaining the details of both the deformed garment and the deformed person. Through extensive experiments on traditional benchmarks, we have quantitatively and qualitatively validated the efficacy of our framework in context of the multi-view try-on task.