Decoding and inference behavior · MLD-1226
Test-time compute
Computation spent after training to improve an answer or action, such as search or repeated sampling.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Use extra verification and candidate generation on hard questions.
Original caveat
More compute can amplify a flawed evaluator or search strategy.
Context and aliases
Level: Intermediate. Stages: Inference · Evaluation · Deployment.
Tags: SLM · LLM · NLP.
Aliases: Inference-time compute.
Related concepts
Concepts linking here
Original references
- Hugging Face: generation strategies ↗ · Reference link
- Fast Inference from Transformers via Speculative Decoding ↗ · Consulted
- vLLM documentation ↗ · Consulted
- Original reference URL ↗