STAM: Zero-Shot Style Transfer Using Diffusion Model via Attention Modulation
Masud An Nur Islam Fahim, Nazmus Saqib, Jani J. Boutellier · 2025
Diffusion models serve as the basis of several different zero-shot image editing applications, including image generation and style transfer. The basic approach in style transfer using diffusion models involves swapping attention componets between the provided content and style images. Straightforward interchange of these components can lead to inadequate style injection or loss of content image characteristics. This paper addresses shortcomings of attention-guided style transfer by two novel contributions: a) preserving content via dual path attention aggregation and b) maintaining the impact of style through modulation of attention components. The proposed STAM approach can provide aesthetically appealing yet contentpreserving style transfer through a combination of these contributions and is also applicable to prompt-driven style transfer. STAM is validated by extensive qualitative and