Systems and efficient execution · MLD-0116
Bfloat16
A 16-bit floating-point format with float32-like exponent range and fewer significand bits.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Train deep networks with broad numeric range and lower memory.
Original caveat
Range and precision are different properties.
Context and aliases
Level: Advanced. Stages: Training · Inference · Deployment.
Tags: DL · LLM · SLM · Systems.
Aliases: BF16.
Related concepts
Concepts linking here
Original references
- PyTorch Fully Sharded Data Parallel ↗ · Consulted
- PyTorch documentation ↗ · Reference link
- vLLM documentation ↗ · Consulted
- Original reference URL ↗