Language-model training and adaptation · MLD-0920
PPO
A policy-gradient family using restricted surrogate updates, commonly with clipping, to limit abrupt policy changes.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Train an RL policy or an RLHF language model.
Original caveat
Clipping is a heuristic constraint, not a universal safety guarantee.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Evaluation.
Tags: DL · SLM · LLM · NLP.
Aliases: Proximal policy optimization.
Related concepts
Concepts linking here
Original references
- Transformers glossary ↗ · Consulted
- PEFT: LoRA conceptual guide ↗ · Consulted
- TRL: supervised fine-tuning ↗ · Reference link
- TRL documentation ↗ · Consulted
- Original reference URL ↗