モデルアーキテクチャ
Autoencoders, VAEs & learned tokenizers
Learn an encoder that compresses data and a decoder that reconstructs it; variational and quantized variants shape the latent space for sampling or downstream generation.
理解のためのモデル
A learned codec. It decides which information survives before another model generates in the compressed space.
データフロー
- Image, audio, or other signal
- Encoder
- Continuous latent or discrete codes
- Optional latent generator
- Decoder reconstruction
学習方法
A reconstruction objective preserves perceptual content. VAEs add a distribution regularizer; vector-quantized models map representations to a discrete codebook and add codebook/commitment objectives.
推論の実行方法
Encode for compression or editing; sample or generate a latent/code sequence; decode to the original modality. Reconstruction quality limits the best possible downstream output.
強み
- Reduces the dimensionality of expensive media
- Creates reusable continuous latents or discrete tokens
- Separates representation learning from the generative prior
トレードオフ
- Compression discards detail
- Latent geometry may not match downstream semantics
- Quantized codebooks can be underused or introduce artifacts
適する場合
- Raw media is too expensive to model directly
- A generative model needs a compact continuous or discrete space
- Reconstruction failures are measured by content slice
避ける・再検討する場合
- Lossless reconstruction is mandatory
- The codec domain differs materially from production data
- Decoder artifacts would be mistaken for generator failures
公開された方式の例
- • Variational Autoencoder (VAE)
- • VQ-VAE discrete visual/audio tokenizers
- • Autoencoders used by latent diffusion
よく組み合わせる要素
Latent diffusionAutoregressive media modelsDiffusion Transformer