Evaluation design and benchmarks · MLD-0383
Evaluator bias
A systematic tendency in an evaluator that affects scores independently of the intended quality criterion.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Check whether a judge favors its own output style.
Original caveat
More evaluations do not remove a systematic bias.
Context and aliases
Level: Intermediate. Stages: Evaluation · Research.
Tags: ML · DL · LLM · SLM · Safety.
Aliases: Judge bias.