Архитектуры моделей
Глоссарий архитектур
Краткие определения терминов из статей о моделях и документации продуктов.
- Attention
- A learned weighted lookup that lets one position combine information from other positions.
- Autoregressive
- Factorizes an output into an ordered sequence and predicts each element from earlier elements.
- Causal mask
- Prevents a token from attending to future tokens during left-to-right language modeling.
- Conditioning
- Information (such as text, an image, a label, or control signal) that guides generation.
- Cross-attention
- Lets one sequence query the hidden states of another sequence or modality.
- Denoising
- Learning or applying a step that estimates clean data or a reverse update from a noisy sample.
- Embedding
- A learned numeric vector used as a representation for similarity or downstream prediction.
- Expert / router
- In MoE layers, an expert is a subnetwork and the router chooses which experts process each token.
- Flow matching
- Learns a time-dependent vector field that transports samples between probability distributions.
- Grounding
- Connecting a model response to supplied, inspectable evidence or an external source of truth.
- KV cache
- Stored attention keys and values from prior tokens, reused during autoregressive decoding.
- Latent
- A learned internal representation, commonly lower-dimensional than the original data.
- Modality projector
- A connector that maps one encoder’s features into the representation expected by another model.
- Reranker
- A second-stage model that scores a small candidate set with richer query–candidate interaction.
- Recurrent state
- A bounded hidden representation carried from one sequence step to the next.
- Token / code
- A discrete unit processed by a sequence model; it may represent text, an image patch, or compressed audio.