SpeedUpNet: A Plug-and-Play Hyper-Network for Accelerating Text-to-Image Diffusion Models

Ant Group

Text-to-image diffusion models exhibit significant advancements while requiring extensive computational resources. To address this, we propose a novel and universal Stable-Diffusion (SD) acceleration module called SpeedUpNet(SUN). SUN can be directly plugged into various SD fine-tuned models without extra training. SUN significantly reduces the number of inference steps to just 4 steps and eliminates the need for classifier-free guidance, and it offers two extra advantages: (1) classifier-free guidance distillation with controllable negative prompts and (2) seamless integration into various fine-tuned Stable-Diffusion models without training.

arXiv Paper  /  GitHub Code