模型架构
Diffusion Transformers (DiT)
Use a Transformer over noisy image or video patches as the denoising network inside a diffusion or flow-based generative process.
思维模型
DiT changes the denoiser backbone; it does not eliminate the diffusion process, latent codec, conditioning, or sampler.
数据流
- Noisy latent
- Patchify + position / time conditioning
- Transformer blocks
- Predicted latent update
- Sampler step → repeat
训练方式
The model learns the same class of denoising, velocity, or flow objective as other diffusion systems while Transformer blocks mix information between latent patches.
推理运行方式
A sampler repeatedly invokes the Transformer at different noise or flow times, then a decoder maps the final latent to media.
优势
- Transformer scaling and implementation ecosystem
- Global interactions between latent patches
- Natural path to shared multimodal token processing
权衡
- Attention cost grows with latent token count
- Still requires repeated denoiser calls
- “DiT” alone does not specify codec, objective, conditioning, or sampler
适用场景
- Transformer scaling infrastructure is available
- Latent patch count and sampling latency fit the budget
- You need a flexible backbone for image or video generation
应避免或质疑的场景
- A small U-Net already meets the objective
- The label is being used as a complete architecture specification
- High-resolution token counts overwhelm the target runtime
已发表的示例系列
- • DiT research family
- • Transformer denoisers inside latent image and video pipelines
常见组合
Latent diffusionAutoencoder / VAEMultimodal conditioningMixture of Experts