Decoding and inference behavior · MLD-1227
Test-time scaling
Studying or exploiting quality changes as inference-time computation increases.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Compare answer accuracy at several search budgets.
Original caveat
Gains are task- and method-dependent, not guaranteed.
Context and aliases
Level: Intermediate. Stages: Inference · Evaluation · Deployment.
Tags: SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Hugging Face: generation strategies ↗ · Reference link
- Fast Inference from Transformers via Speculative Decoding ↗ · Consulted
- vLLM documentation ↗ · Consulted
- Original reference URL ↗