AnyEnhance: A Unified Generative Model With Prompt-Guidance and Self-Critic for Voice Enhancement
J. S. Zhang, Jing Yang, Zihao Fang, Yuancheng Wang, Zehua Zhang, Zhuo Wang, Fan Fan, Zhizheng Wu · IEEE Transactions on Audio Speech and Language Processing · 2025
We introduceAnyEnhance, a unified generative model for voice enhancement that processes both speech and singing voices. Based on a masked generative model,AnyEnhanceis capable of handling both speech and singing voices, supporting a wide range of enhancement tasks including denoising, dereverberation, declipping, super-resolution, and target speaker extraction, all simultaneously and without fine-tuning.AnyEnhanceintroduces a prompt-guidance mechanism for in-context learning, which allows the model to natively accept a reference speaker’s timbre. In this way, it could boost enhancement performance when a reference audio is available and enable the target speaker extraction task without altering the underlying architecture. Moreover, we also introduce a self-critic mechanism into the generative process for masked generative models, yielding higher-quality outputs through iterative self-assessment and refinement. Extensive experiments on various enhancement tasks demonstrateAnyEnhanceoutperforms existing methods in terms of both objective metrics and subjective listening tests.