Data and feature engineering · MLD-0797
Near-duplicate detection
Finding records that are highly similar without being identical.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Detect lightly edited copies of documents.
Original caveat
Similarity thresholds trade false matches against missed duplicates.
Context and aliases
Level: Intermediate. Stages: Data · Training · Evaluation.
Tags: ML · DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- scikit-learn: cross-validation ↗ · Reference link
- scikit-learn: missing-value imputation ↗ · Reference link
- scikit-learn: preprocessing ↗ · Reference link
- scikit-learn: feature selection ↗ · Reference link
- Original reference URL ↗