Model Architectures
Retrieval-augmented & hybrid systems
Retrieve evidence at request time and provide it to a generator, classifier, or agent instead of relying only on model parameters.
Mental model
Open-book inference: retrieval chooses the evidence; generation uses it. Either half can fail independently.
Data flow
- Ingest → chunk → index
- User query
- Dense, sparse, graph, or hybrid retrieval
- Rerank + context assembly
- Model answer with evidence links
How it trains
Components may be trained separately or jointly: dual encoders learn retrieval, rerankers learn pairwise relevance, and generators learn grounded response behavior. Many practical RAG systems use pretrained components without end-to-end training.
How inference runs
A query may be rewritten or routed; multiple retrievers produce candidates; filters and a reranker select evidence; the model answers from a bounded context. Citation correctness requires checking that claims are actually supported.
Strengths
- Fresh, private, or attributable knowledge without weight updates
- Evidence can be inspected and access-controlled
- Indexes and generators can evolve independently
Trade-offs
- Bad chunking or retrieval creates confident answers from missing evidence
- Ingestion freshness, permissions, and deletion become production systems
- Retrieved content can carry prompt injection or poisoned data
Use it when
- Answers depend on changing or proprietary sources
- Users need traceable evidence
- Recall, ranking, grounding, and answer quality can be evaluated separately
Avoid or challenge it when
- The corpus is tiny enough for a deterministic lookup
- No trustworthy source exists
- Adding a vector database is being treated as a complete RAG design
Illustrative published families
- • Original Retrieval-Augmented Generation research architecture
- • Dense + sparse hybrid retrieval with reranking
- • Graph or tool-based retrieval pipelines
Commonly combines with
Embedding modelsDual encodersRerankersText / chat modelsTool systems