Modellarchitekturen
Text & chat language models
Generate text one token at a time, usually with a causal decoder trained for next-token prediction and then adapted to follow instructions.
Denkmodell
A probabilistic continuation engine wrapped in a conversation protocol, not a database, search engine, or deterministic rules engine.
Datenfluss
- Messages, documents, or code
- Tokenizer + role/tool markers
- Causal language-model backbone
- Next-token distribution
- Decode until a stop condition
So wird trainiert
Pretraining commonly minimizes next-token cross-entropy over large token sequences. Instruction tuning, preference optimization, safety training, and tool-use examples then shape interaction behavior; they do not change the basic need for evidence and evaluation.
So läuft die Inferenz
The model repeatedly predicts and samples or selects one next token. A KV cache reuses prior attention state; long answers remain serial, and decoding settings change variability rather than factuality.
Stärken
- Open-ended writing, transformation, explanation, code, and conversation
- Learns tasks from instructions and examples in context
- Can emit structured tool calls or schemas when constrained and validated
Zielkonflikte
- Can produce fluent unsupported claims
- Autoregressive output adds per-token latency
- Context, prompting, and sampling choices materially affect behavior
Geeignet, wenn
- The task needs flexible language generation or synthesis
- A rubric and representative evaluation set can define acceptable behavior
- Fresh facts can be grounded through retrieval or tools
Vermeiden oder hinterfragen, wenn
- A deterministic parser, query, or rules engine solves the task reliably
- Exact current facts are required but no trusted source is connected
- Unreviewed output could directly trigger a high-impact action
Beispielhafte veröffentlichte Familien
- • GPT-style causal language models
- • Instruction-tuned descendants such as the InstructGPT research system
Häufig kombiniert mit
Decoder-only TransformerMixture of ExpertsRetrieval-augmented systemsTool & reasoning systems