Model Architectures
Embedding models
Map text, images, audio, users, or items into vectors whose geometry is trained to preserve a useful notion of similarity.
Mental model
A learned coordinate system. Nearness means “similar under the training objective,” not universally equivalent or factually related.
Data flow
- Input item
- Encoder
- Pooling or projection
- Fixed-length vector
- Similarity search, clustering, or classifier
How it trains
Contrastive, metric-learning, classification, or paired-data objectives pull useful matches together and push negatives apart. The negative-sampling strategy and domain data define what similarity means.
How inference runs
Each item is encoded in one forward pass. Stored vectors enable fast approximate-nearest-neighbor search; a cross-encoder can rerank the small candidate set for finer interaction.
Strengths
- Efficient semantic retrieval over large collections
- Reusable features for clustering, routing, recommendations, and deduplication
- Candidate vectors can be computed and indexed ahead of time
Trade-offs
- A single vector compresses away token-level detail
- Similarity degrades under domain, language, or time drift
- Scores are model- and index-specific, not calibrated probabilities
Use it when
- You need semantic candidate retrieval
- The corpus is too large for pairwise scoring
- You can evaluate recall on production-shaped queries
Avoid or challenge it when
- Exact lexical matching is the only requirement
- You need a generated response rather than a representation
- A similarity threshold would be deployed without calibration
Illustrative published families
- • BERT-derived sentence encoders
- • The separate text and image towers used by CLIP
Commonly combines with
Encoder-only TransformerContrastive / dual encodersRetrieval-augmented systems