Model Architectures
Contrastive & dual encoders
Encode two inputs independently (such as a query and document or image and caption) and train matching pairs to land near each other.
Mental model
Two readers meet in a shared coordinate system; they are fast because candidates do not interact until their vectors are compared.
Data flow
- Paired inputs A and B
- Separate or shared encoders
- Projected normalized vectors
- Similarity matrix
- Contrastive matching loss
How it trains
Batch contrastive objectives reward paired examples and treat other examples as negatives. Negative quality, duplicate semantics, and temperature materially affect the learned space.
How inference runs
Encode each side independently and compare vectors with dot product or cosine similarity. Precompute the large candidate side; optionally rerank top results with a cross-encoder.
Strengths
- Scales retrieval to large corpora
- Aligns modalities without a joint decoder
- Enables zero-shot classification through label text in some settings
Trade-offs
- Independent encoding misses fine-grained cross-input interactions
- False negatives can distort training
- Global similarity may ignore spatial, temporal, or compositional details
Use it when
- Fast retrieval or matching is the first stage
- One side can be indexed offline
- Recall is followed by task-appropriate reranking when needed
Avoid or challenge it when
- Every candidate needs deep token-to-token comparison
- The task depends on precise spatial relationships
- Raw similarity scores are assumed to be probabilities
Illustrative published families
- • CLIP image–text dual encoder
- • Dense passage retrieval
- • Bi-encoder semantic search
Commonly combines with
Embedding modelsVision encodersRAGCross-encoder rerankers