Архитектуры моделей
Latent diffusion
Run diffusion in an autoencoder’s lower-dimensional latent space, then decode the generated representation back to pixels or another signal.
Ментальная модель
Denoise a compact learned sketch instead of every raw pixel.
Поток данных
- Training media
- Frozen or jointly trained autoencoder latent
- Conditional latent denoiser
- Sampled clean latent
- Decoder → media
Как проходит обучение
An autoencoder first learns a perceptual compression. The diffusion model then learns the denoising objective over those latents, often conditioned through cross-attention.
Как выполняется инференс
Sample in the smaller latent tensor over several steps and decode once. Image-to-image and inpainting can begin from encoded and selectively noised source content.
Сильные стороны
- Lower denoising cost than raw high-resolution pixels
- Modular text conditioning and editing
- Reusable autoencoder and generator components
Компромиссы
- The autoencoder creates a reconstruction ceiling
- Fine text or high-frequency detail can be lost in compression
- Two-stage failures are harder to attribute
Использовать, когда
- High-resolution media generation must fit practical compute
- Editing and conditional control are required
- Codec reconstruction is validated on the target domain
Избегать или пересмотреть, когда
- The autoencoder drops task-critical details
- End-to-end simplicity matters more than generation cost
- Latent-space shortcuts could hide safety-relevant content
Примеры опубликованных семейств
- • Latent Diffusion Models
- • Stable Diffusion research lineage
Часто сочетается с
Autoencoder / VAEU-Net denoiserDiffusion TransformerText encoder