Архитектуры генеративных моделей

Generate text one token at a time, usually with a causal decoder trained for next-token prediction and then adapted to follow instructions.

Map text, images, audio, users, or items into vectors whose geometry is trained to preserve a useful notion of similarity.

Synthesize or transform pixels, most often through a text or image conditioner, a latent generator, and an image decoder.

Generate spatial and temporal structure together using frame, patch, or latent video representations with cross-frame computation.

A family of task-specific pipelines: speech recognition, speech synthesis, audio understanding, voice conversion, sound generation, and music generation are not one architecture.

Let every input token attend to tokens on both sides, producing contextual representations rather than an open-ended generation loop.

Use a causal mask so each position sees only earlier tokens, matching the left-to-right generation process used by most general-purpose chat models.

Encode an input bidirectionally, then generate an output causally while cross-attending to the encoded source.

Replace selected dense sublayers with many expert networks and a learned router that activates only a small subset for each token.

Update a compact state as tokens arrive instead of retaining an explicit attention relationship between every pair of positions.

Learn an encoder that compresses data and a decoder that reconstructs it; variational and quantized variants shape the latent space for sampling or downstream generation.

Train a generator to fool a discriminator while the discriminator learns to distinguish generated samples from training data.

Transform a simple distribution into a complex data distribution through a sequence of invertible mappings with tractable Jacobians.

Learn to reverse a gradual noising process, producing data by repeatedly transforming noise toward a sample.

Run diffusion in an autoencoder’s lower-dimensional latent space, then decode the generated representation back to pixels or another signal.

Use a Transformer over noisy image or video patches as the denoising network inside a diffusion or flow-based generative process.

Convert images, video, or audio into discrete codes and predict those codes in an ordering, often with a causal Transformer.

Transform pixels into feature maps, patch tokens, or pooled vectors that downstream classifiers, retrievers, decoders, or multimodal models can consume.

Encode two inputs independently (such as a query and document or image and caption) and train matching pairs to land near each other.

Connect modality-specific encoders and generators through projection, cross-attention, shared token spaces, or combinations of these mechanisms.

Retrieve evidence at request time and provide it to a generator, classifier, or agent instead of relying only on model parameters.

Place a model inside a controller that can plan, call typed tools, inspect results, revise, verify, stop, and hand control to a person.