Enable Fast Sampling for Seq2Seq Text Diffusion
Pan Liu, Xiaohua Tian, Zhouhan Lin · 2024
Diffusion models exhibit promising capacity for generating high-quality text.However, owing to the curved nature of the generation path, they necessitate traversing numerous steps to guarantee high text quality.In this paper, we propose an efficient model FMSeq 1 , which utilizes flow matching to straighten the generation path, thereby enabling fast sampling for diffusion-based seq2seq text generation.Specifically, we construct transport flow only on the target sequences to adapt the diffusionbased model to flow matching.Furthermore, we explore different settings and identify targetparameterization, self-conditioning, and timedifference as three effective techniques to improve the generation quality under a few steps.Experiments on four popular tasks demonstrate that FMSeq generates texts of comparable quality to the SOTA diffusion-based DiffuSeq in just 10 steps, achieving a 200-fold speedup.