Diffusion models (Stable Diffusion, Midjourney, video models) generate images by starting from pure noise and removing it step by step until a picture emerges — guided by your text prompt.
Train forward, generate backward
- Forward (training): take real images and progressively add noise until they're static. The model learns what each noisy step looked like.
- Reverse (generation): start from random noise and apply the learned denoiser repeatedly, nudged toward your prompt, until a clean image forms.
noise → denoise step 1 → … → denoise step N → image
(guided by the text prompt at every step)Key pieces
| Part | Role |
|---|---|
| U-Net / DiT | The denoiser network |
| Text encoder | Turns your prompt into guidance (embeddings) |
| VAE | Compresses to a latent space so it's tractable |
| Scheduler | How many steps, how much noise per step |
Diffusion vs LLMs
An LLM builds text left-to-right, one token at a time. A diffusion model refines the whole image at once, sharpening it over many passes. Different shape, same idea: learn the data, then generate from it.
Run them locally with Local Image Generation with ComfyUI.
Related: Local Image Generation with ComfyUI · How Transformers Work