模型架构
Contrastive & dual encoders
Encode two inputs independently (such as a query and document or image and caption) and train matching pairs to land near each other.
思维模型
Two readers meet in a shared coordinate system; they are fast because candidates do not interact until their vectors are compared.
数据流
- Paired inputs A and B
- Separate or shared encoders
- Projected normalized vectors
- Similarity matrix
- Contrastive matching loss
训练方式
Batch contrastive objectives reward paired examples and treat other examples as negatives. Negative quality, duplicate semantics, and temperature materially affect the learned space.
推理运行方式
Encode each side independently and compare vectors with dot product or cosine similarity. Precompute the large candidate side; optionally rerank top results with a cross-encoder.
优势
- Scales retrieval to large corpora
- Aligns modalities without a joint decoder
- Enables zero-shot classification through label text in some settings
权衡
- Independent encoding misses fine-grained cross-input interactions
- False negatives can distort training
- Global similarity may ignore spatial, temporal, or compositional details
适用场景
- Fast retrieval or matching is the first stage
- One side can be indexed offline
- Recall is followed by task-appropriate reranking when needed
应避免或质疑的场景
- Every candidate needs deep token-to-token comparison
- The task depends on precise spatial relationships
- Raw similarity scores are assumed to be probabilities
已发表的示例系列
- • CLIP image–text dual encoder
- • Dense passage retrieval
- • Bi-encoder semantic search
常见组合
Embedding modelsVision encodersRAGCross-encoder rerankers