Systems and efficient execution · MLD-0600
Kernel fusion
Combining multiple operations into fewer kernels to reduce launches and intermediate memory traffic.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Fuse elementwise operations around a matrix multiplication.
Original caveat
Fusion benefits depend on hardware and operation structure.
Context and aliases
Level: Advanced. Stages: Training · Inference · Deployment.
Tags: DL · LLM · SLM · Systems.
Aliases: None recorded.
Related concepts
Concepts linking here
Technology connections
- Apache TVM — ML compilation and optimization stack.
- CUDA — NVIDIA GPU programming platform and libraries.
- cuDNN — NVIDIA library of neural-network computation primitives.
- CUTLASS — CUDA templates and abstractions for high-performance linear algebra.
- FlashAttention — Memory-aware exact attention kernel implementations.
- FlashInfer — GPU kernels for LLM inference operations.
- IREE — MLIR-based compiler and runtime for ML deployment.
- NCCL — NVIDIA collective-communication library.
- OpenXLA — Compiler infrastructure for ML computation across supported hardware.
- ROCm — AMD GPU computing software stack.
- torch.compile — PyTorch graph capture and compiler entry point.
- Triton GPU language — Python-embedded language and compiler for GPU kernels.
Original references
- PyTorch Fully Sharded Data Parallel ↗ · Consulted
- PyTorch documentation ↗ · Reference link
- vLLM documentation ↗ · Consulted
- Original reference URL ↗