Sequence architectures and attention · MLD-1252
Total parameters
All stored model parameters, including inactive experts and other components.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Measure total model storage requirements.
Original caveat
Reported counts may include or exclude embeddings or shared weights.
Context and aliases
Level: Intermediate. Stages: Training · Inference · Theory.
Tags: DL · SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Attention Is All You Need ↗ · Consulted
- Transformers glossary ↗ · Consulted
- PyTorch neural-network layers ↗ · Reference link
- Original reference URL ↗