Neural-network building blocks · MLD-0922
Pre-activation
The value entering an activation function, often an affine layer output.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Inspect values before ReLU clipping.
Original caveat
Terminology can differ for complex blocks.
Context and aliases
Level: Intermediate. Stages: Training · Inference · Theory.
Tags: DL · SLM · LLM · Vision · NLP.
Aliases: None recorded.
Related concepts
Original references
- Deep Learning: online textbook ↗ · Consulted
- PyTorch neural-network layers ↗ · Reference link
- PyTorch documentation ↗ · Reference link
- Original reference URL ↗