Optimization and training dynamics · MLD-0477
Gradient clipping
Limiting gradient magnitude or individual values before updating parameters.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Prevent rare extreme gradients from destabilizing training.
Original caveat
Clipping can mask underlying instability if used without diagnosis.
Context and aliases
Level: Intermediate. Stages: Training · Theory.
Tags: ML · DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Deep Learning: online textbook ↗ · Consulted
- PyTorch optimization algorithms ↗ · Reference link
- PyTorch documentation ↗ · Reference link
- Original reference URL ↗