Autoencoders, VAEs & learned tokenizers

How it trains

A reconstruction objective preserves perceptual content. VAEs add a distribution regularizer; vector-quantized models map representations to a discrete codebook and add codebook/commitment objectives.

How inference runs

Encode for compression or editing; sample or generate a latent/code sequence; decode to the original modality. Reconstruction quality limits the best possible downstream output.

Strengths

• •Reduces the dimensionality of expensive media
• •Creates reusable continuous latents or discrete tokens
• •Separates representation learning from the generative prior

Trade-offs

• •Compression discards detail
• •Latent geometry may not match downstream semantics
• •Quantized codebooks can be underused or introduce artifacts

Use it when

• Raw media is too expensive to model directly
• A generative model needs a compact continuous or discrete space
• Reconstruction failures are measured by content slice

Avoid or challenge it when

• Lossless reconstruction is mandatory
• The codec domain differs materially from production data
• Decoder artifacts would be mistaken for generator failures