Decoding and inference behavior · MLD-1072
Sampling
Randomly selecting outputs according to a specified distribution.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Generate diverse continuations from token probabilities.
Original caveat
Sampling randomness does not create calibrated uncertainty by itself.
Context and aliases
Level: Intermediate. Stages: Inference · Evaluation · Deployment.
Tags: SLM · LLM · NLP.
Aliases: Stochastic decoding.
Related concepts
Concepts linking here
Original references
- Hugging Face: generation strategies ↗ · Reference link
- Fast Inference from Transformers via Speculative Decoding ↗ · Consulted
- vLLM documentation ↗ · Consulted
- Original reference URL ↗