Language-model training and adaptation · MLD-0155
Causal language modeling
Training to predict each token from allowed preceding context.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Train an autoregressive decoder.
Original caveat
Causal refers to token order, not causal reasoning.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Evaluation.
Tags: DL · SLM · LLM · NLP.
Aliases: CLM · next-token prediction.
Related concepts
Concepts linking here
Original references
- Transformers glossary ↗ · Consulted
- PEFT: LoRA conceptual guide ↗ · Consulted
- TRL: supervised fine-tuning ↗ · Reference link
- TRL documentation ↗ · Consulted
- Original reference URL ↗