Language-model training and adaptation · MLD-0680
LoRA alpha
A scaling hyperparameter controlling the magnitude of a LoRA update under its chosen convention.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Scale an adapter relative to base weights.
Original caveat
Scaling formulas differ for standard and rank-stabilized LoRA.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Evaluation.
Tags: DL · SLM · LLM · NLP.
Aliases: Adapter scaling.
Related concepts
Concepts linking here
Original references
- Transformers glossary ↗ · Consulted
- PEFT: LoRA conceptual guide ↗ · Consulted
- TRL: supervised fine-tuning ↗ · Reference link
- TRL documentation ↗ · Consulted
- Original reference URL ↗