Enhancing Conditional Diffusion Model Robustness in Inference with Optimal Combination Factor
Weifeng Xu, Xiang Zhu, Xiaoyong Li · 2025
Web-oriented services such as image generation and weather forecasting increasingly rely on pre-trained conditional diffusion models (CDMs). When deployed in the open Internet, however, CDMs suffer performance degradation in inference phrase due to noisy inputs, revealing limited robustness. Existing robustness-enhancement techniques such as adversarial training and adversarial purification are not suitable to CDMs. Moreover, those methods fail to meet real-time and reliability requirements of web services. To bridge this gap, we introduce a lightweight, plug-and-play method that can be integrated into any CDMs inference pipeline without altering the underlying architecture. Inspired by deep reinforcement learning, the approach adaptively fuses the outputs of two neural networks, calculating an optimal combination factor via a control-theoretic optimization scheme. Experimental results on CIFAR-10, Moving MNIST and SEVIR datasets demonstrate that CDMs augmented with our method produce higher-quality generation and more accurate forecasts across multiple noise levels. The proposed strategy provides a practical pathway toward the robust deployment of generative AI in safety-critical web applications.