Systems and efficient execution · MLD-0472
GPU
A processor designed for highly parallel workloads, widely used for tensor computation.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Accelerate neural-network training.
Original caveat
Available memory and supported operations matter alongside compute speed.
Context and aliases
Level: Foundational. Stages: Training · Inference · Deployment.
Tags: DL · LLM · SLM · Systems.
Aliases: Graphics processing unit.
Related concepts
Concepts linking here
Technology connections
- Amazon SageMaker AI — AWS managed ML development, training and deployment services.
- Apache TVM — ML compilation and optimization stack.
- Azure Machine Learning — Azure service for managing ML training and deployment workflows.
- CUDA — NVIDIA GPU programming platform and libraries.
- cuDNN — NVIDIA library of neural-network computation primitives.
- CUTLASS — CUDA templates and abstractions for high-performance linear algebra.
- Databricks — Managed data/analytics and ML platform.
- FlashAttention — Memory-aware exact attention kernel implementations.
- FlashInfer — GPU kernels for LLM inference operations.
- Hugging Face Inference Endpoints — Managed deployment service for supported model endpoints.
- Hugging Face Inference Providers — Unified access to supported hosted inference providers.
- IREE — MLIR-based compiler and runtime for ML deployment.
- Kubeflow — Kubernetes-native ML platform components.
- Modal — Managed platform for Python compute and deployment.
- NCCL — NVIDIA collective-communication library.
- OpenXLA — Compiler infrastructure for ML computation across supported hardware.
- ROCm — AMD GPU computing software stack.
- Runpod — GPU compute and serverless deployment platform.
- SkyPilot — Tooling for running AI workloads across supported clouds and clusters.
- Slurm — Workload manager for HPC clusters.
- torch.compile — PyTorch graph capture and compiler entry point.
- Triton GPU language — Python-embedded language and compiler for GPU kernels.
Original references
- PyTorch Fully Sharded Data Parallel ↗ · Consulted
- PyTorch documentation ↗ · Reference link
- vLLM documentation ↗ · Consulted
- Original reference URL ↗