Decoding and inference behavior · MLD-0106
Beam width
The number of partial hypotheses retained in beam search.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Compare decoding with four versus eight beams.
Original caveat
Larger beams do not guarantee better task-level quality.
Context and aliases
Level: Intermediate. Stages: Inference · Evaluation · Deployment.
Tags: SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Original references
- Hugging Face: generation strategies ↗ · Reference link
- Fast Inference from Transformers via Speculative Decoding ↗ · Consulted
- vLLM documentation ↗ · Consulted
- Original reference URL ↗