モデルアーキテクチャ
Contrastive & dual encoders
Encode two inputs independently (such as a query and document or image and caption) and train matching pairs to land near each other.
理解のためのモデル
Two readers meet in a shared coordinate system; they are fast because candidates do not interact until their vectors are compared.
データフロー
- Paired inputs A and B
- Separate or shared encoders
- Projected normalized vectors
- Similarity matrix
- Contrastive matching loss
学習方法
Batch contrastive objectives reward paired examples and treat other examples as negatives. Negative quality, duplicate semantics, and temperature materially affect the learned space.
推論の実行方法
Encode each side independently and compare vectors with dot product or cosine similarity. Precompute the large candidate side; optionally rerank top results with a cross-encoder.
強み
- Scales retrieval to large corpora
- Aligns modalities without a joint decoder
- Enables zero-shot classification through label text in some settings
トレードオフ
- Independent encoding misses fine-grained cross-input interactions
- False negatives can distort training
- Global similarity may ignore spatial, temporal, or compositional details
適する場合
- Fast retrieval or matching is the first stage
- One side can be indexed offline
- Recall is followed by task-appropriate reranking when needed
避ける・再検討する場合
- Every candidate needs deep token-to-token comparison
- The task depends on precise spatial relationships
- Raw similarity scores are assumed to be probabilities
公開された方式の例
- • CLIP image–text dual encoder
- • Dense passage retrieval
- • Bi-encoder semantic search
よく組み合わせる要素
Embedding modelsVision encodersRAGCross-encoder rerankers