Loading patterns…
Odds Ratio Preference Optimization(ORPO)
Reference-free preference alignment that adds an odds-ratio penalty for rejected responses to the chosen-response language-modeling objective
In 30 seconds
- What
- Trains a model on chosen responses while penalizing rejected responses via odds-ratio loss, skipping the reference model entirely.
- When to use
- You have paired preference data and want faster, cheaper alignment without maintaining a separate reference model.
- Watch out
- Odds-ratio penalty can destabilize training if lambda is mistuned; requires careful hyperparameter search to avoid capability loss.
Ask the AI expert about this pattern
Opens the assistant with your question prefilled. You review it before sending.
Odds Ratio Preference Optimization: Overview
Reference-free preference alignment that adds an odds-ratio penalty for rejected responses to the chosen-response language-modeling objective
- Reference-model-free training
- Chosen-response SFT objective
- Odds-ratio preference penalty
- Single training phase
- Paired preference data
- Preference and capability evaluation
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.
References
The papers, specifications, and repositories this pattern is based on.
From the engineer behind this catalog
Pay less per call with a model you own
If one repetitive task drives your LLM bill, we can distill it into a small open-weight model trained on your own traffic, then test it against your current model on examples it never saw. Quality and per-call cost come back side by side, in writing.
€2,900 per task, fixed price. You pay only if it matches your current model on your own held-out examples.