Neural-network building blocks · MLD-0636
Leaky ReLU
A ReLU variant with a small nonzero negative-side slope.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Reduce permanently inactive rectifier units.
Original caveat
The negative slope is usually fixed.
Context and aliases
Level: Intermediate. Stages: Training · Inference · Theory.
Tags: DL · SLM · LLM · Vision · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Deep Learning: online textbook ↗ · Consulted
- PyTorch neural-network layers ↗ · Reference link
- PyTorch documentation ↗ · Reference link
- Original reference URL ↗