Optimization and training dynamics · MLD-0650
Linear decay
A schedule reducing a quantity at a constant rate over a defined interval.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Decrease learning rate to zero over training.
Original caveat
The end point and step-count convention matter.
Context and aliases
Level: Intermediate. Stages: Training · Theory.
Tags: ML · DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Original references
- Deep Learning: online textbook ↗ · Consulted
- PyTorch optimization algorithms ↗ · Reference link
- PyTorch documentation ↗ · Reference link
- Original reference URL ↗