Agent Engineering Glossary
The vocabulary, defined plainly. Most of these terms are looked up because two of them are being conflated, so each entry says what the term is not as well as what it is.
Agents
- Agent
- A program that calls a model in a loop, executes the tools the model asks for, feeds the results back, and stops when the model answers without requesting a tool. The loop is the agent; everything a framework adds around it is scaffolding. More
- Agentic
- Describes a system where the model decides what happens next, rather than following a path fixed in code. The useful test is whether removing the model would leave a working program: if it would, the system is automated, not agentic.
- ReActalso: reason and act
- Interleaving reasoning with tool calls so the model can act on what it learns mid-task, instead of planning everything upfront. The prebuilt agent in most frameworks is a ReAct loop. More
- Tool callingalso: function calling
- The model returns a structured request to run a named function with arguments, rather than prose. The runtime executes it and returns the result as another message. The model never runs anything itself. More
- Supervisoralso: orchestrator, manager agent
- A model that chooses which worker runs next and what it is told to do, instead of that order being fixed in code. Costs one model call per step before any work happens, and is worth it only when the path genuinely varies. More
- Subagent
- An agent invoked by another agent, usually with its own context window and a narrower brief. The point is context isolation: the parent gets a summarised result instead of the whole transcript.
- Handoff
- Transferring control of a conversation from one agent to another, with the receiving agent taking over the interaction rather than returning a result. Distinct from a subagent call, which comes back. More
- Step budgetalso: max iterations
- The cap on how many times an agent loop may run before it is stopped. Not a tuning knob but a safety feature: without it, a model repeatedly calling a failing tool will keep calling it until something else intervenes.
Context and memory
- Context window
- The maximum number of tokens a model can attend to in one request, counting the prompt and the generated output together. A ceiling, not an allowance: attention cost grows with sequence length, so filling it is something you pay for. More
- Token
- The unit a model reads and bills in, roughly a common word fragment. English prose averages about four characters per token; code and non-Latin scripts are denser, so character-based estimates mislead there.
- System prompt
- Instructions placed before the conversation to set behaviour, format and constraints. It is input like any other, which is why it can be overridden by sufficiently determined text later in the context.
- Few-shot
- Including worked examples in the prompt so the model infers the pattern, rather than describing the pattern in words. Often outperforms a longer instruction, and costs input tokens on every single call.
- KV cache
- Stored attention keys and values for tokens already processed, so a continuation does not recompute the whole prefix. This is why a stable prompt prefix is cheaper than one that changes at the start. More
- Prompt caching
- Provider-side reuse of the computed prefix of a repeated prompt, billed at a reduced rate. Requires the cached portion to be byte-identical and to come first, which constrains how you order a prompt.
- Context rotalso: context pollution
- Degradation of an agent as its transcript grows: the original goal is buried under tool output, and the model starts responding to the noise. The usual cause of a demo that works and a production run that does not.
- Checkpointer
- Storage that snapshots agent state after each step so a run can resume, pause for a human, or be rewound. Cheap to build and expensive to run: snapshotting a growing message list every step writes bytes proportional to the square of the step count. More
Retrieval and knowledge
- RAGalso: retrieval-augmented generation
- Fetching relevant documents at query time and putting them in the context, so the model answers from supplied text rather than memory. The right fix for facts that change; the wrong fix for behaviour or format. More
- Embedding
- A vector representation of text where distance approximates similarity, used to retrieve passages by meaning rather than keyword. Similarity is not relevance: two texts can be close in vector space and useless to each other.
- Chunking
- Splitting documents into retrievable pieces. The size is a real tradeoff: small chunks retrieve precisely and lose surrounding context, large ones carry context and dilute the match.
- Grounding
- Constraining an answer to supplied source material, and ideally citing it. Reduces fabrication but does not remove it: a model can still misread or over-generalise a source it was given.
- Hallucinationalso: confabulation
- A fluent, confident output that is not supported by the input or by fact. Not a bug to be patched but a property of generation, which is why systems that matter are built to verify rather than to trust.
Protocols and interfaces
- MCPalso: Model Context Protocol
- An open protocol for exposing tools, resources and prompts to model applications over JSON-RPC. Since the 2026-07-28 revision it has no handshake: every request carries its own protocol version, identity and capabilities. More
- JSON Schema
- The format tool parameters are described in, and what the model reads to decide what to send. It describes what should arrive, not what will: arguments still need validating as untrusted input.
- Structured output
- Constraining generation to a schema so the result parses reliably. Removes a class of parsing failure and does not make the content correct, only well-formed. More
- A2Aalso: agent-to-agent
- Protocols for agents built by different parties to discover each other and exchange tasks, as opposed to MCP, which connects one application to its tools. More
Training and adaptation
- Fine-tuning
- Further training a pretrained model on your own examples to change its behaviour, format or voice. It does not install facts that change, and it cannot rescue a task you have no way to evaluate. More
- LoRAalso: low-rank adaptation
- Training small adapter matrices while the base weights stay frozen, cutting memory and producing an artifact of tens of megabytes rather than a full model copy. The default parameter-efficient method. More
- QLoRA
- LoRA over a quantised base model, so a larger model fits in less memory. Hardware-sensitive: the 4-bit path needs a GPU generation that supports it, which rules out older cards. More
- Distillation
- Training a smaller model on a larger one’s outputs to approach its quality at lower cost and latency. A cost play once quality is acceptable, not a way to reach quality.
- Quantisationalso: quantization
- Storing weights at lower numeric precision to cut memory and speed up inference, at some accuracy cost. Distinct from distillation: the model is the same size in parameters, just cheaper per parameter. More
- Contamination
- Training data containing your evaluation data, which makes the eval score measure memorisation. Detected cheaply by n-gram containment, though that only catches copied text, never facts the model learned another way. More
- Catastrophic forgetting
- Losing general capability while acquiring a narrow one during fine-tuning. The reason a model that got better at your task can get worse at everything around it, and the reason to keep a broad eval alongside the specific one. More
Evaluation and operations
- Evalalso: evaluation set
- A fixed set of inputs with expected behaviour, run before and after a change. Without one you cannot distinguish improvement from regression, and every intervention becomes a guess. More
- LLM-as-judge
- Using a model to score another model’s output against a rubric. Scales where human review cannot, and inherits the judge’s biases, including a documented preference for its own style and for longer answers. More
- Leakage
- Information reaching a model that would not be available at inference time, typically near-duplicate rows straddling a train/test split. Produces an eval score that measures memorisation and collapses in production.
- Guardrail
- A check around a model that blocks or rewrites unacceptable input or output. Sits outside the model because the model cannot be relied on to police itself, and is defeated by anything that makes the harmful path look acceptable. More
- Prompt injection
- Text from an untrusted source that the model reads as instruction rather than data. The delivery route is anything model-facing: a web page, a document, a tool description, a tool result. More
- Tool poisoning
- Prompt injection delivered through a tool definition, where a description read by the model carries instructions. Needs no exploit: the protocol is working exactly as designed. More
- Time to first tokenalso: TTFT
- Latency from request to the first streamed token, as opposed to total generation time. What a user perceives as responsiveness, and what streaming exists to improve. More
- Throughput
- Tokens produced per unit of time across all concurrent requests. Trades against per-request latency: batching raises throughput and makes any individual answer slower to arrive. More
For the architectures behind these models rather than the vocabulary around them, see the model architecture glossary.