Language-model training and adaptation · MLD-0338
DPO
A preference-training objective using relative policy/reference log probabilities to learn directly from preferred response pairs.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Adapt a model without separately fitting a reward model in the basic method.
Original caveat
It is not synonymous with every preference-learning method.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Evaluation.
Tags: DL · SLM · LLM · NLP.
Aliases: Direct preference optimization.
Related concepts
Concepts linking here
Original references
- Direct Preference Optimization ↗ · Consulted
- TRL documentation ↗ · Consulted
- Original reference URL ↗