Distillation and model compression · MLD-0754
Model compression
Reducing model storage, computation, or deployment resource requirements.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Combine pruning, quantization, and distillation.
Original caveat
Compression can change accuracy, calibration, and robustness.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Deployment.
Tags: DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Distilling the Knowledge in a Neural Network ↗ · Consulted
- PEFT: LoRA conceptual guide ↗ · Consulted
- PEFT: quantization guide ↗ · Consulted
- Original reference URL ↗