Neural-network building blocks · MLD-1036
Reverse-mode autodiff
Automatic differentiation propagating sensitivities from outputs back to inputs.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Differentiate one scalar loss with many parameters.
Original caveat
It typically stores or recomputes intermediate activations.
Context and aliases
Level: Intermediate. Stages: Training · Inference · Theory.
Tags: DL · SLM · LLM · Vision · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Deep Learning: online textbook ↗ · Consulted
- PyTorch neural-network layers ↗ · Reference link
- PyTorch documentation ↗ · Reference link
- Original reference URL ↗