Mathematics and probability · MLD-0791
Mutual information
The reduction in uncertainty about one variable provided by another.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Select features informative about the target.
Original caveat
Estimating it reliably can be difficult in high dimensions.
Context and aliases
Level: Foundational. Stages: Theory · Training · Evaluation.
Tags: ML · DL · SLM · LLM.
Aliases: MI.
Related concepts
Concepts linking here
Original references
- Deep Learning: online textbook ↗ · Consulted
- Mathematics for Machine Learning ↗ · Reference link
- Original reference URL ↗