Systems and efficient execution · MLD-0012
Activation checkpointing
Saving fewer intermediate activations and recomputing them during backpropagation to reduce memory use.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Fit longer sequences in GPU memory.
Original caveat
It trades additional computation for lower activation storage.
Context and aliases
Level: Advanced. Stages: Training · Inference · Deployment.
Tags: DL · LLM · SLM · Systems.
Aliases: Gradient checkpointing · Rematerialization.
Related concepts
Original references
- PyTorch Fully Sharded Data Parallel ↗ · Consulted
- PyTorch documentation ↗ · Reference link
- vLLM documentation ↗ · Consulted
- Original reference URL ↗