Essay

Selecting the Human Benchmark for a Performance Target

Question: In a concise analytical response, explain why a 2% miss rate from the specialist review panel is a better basis than 5% or 10% when estimating the human benchmark and setting a target level.

Sample answer: The panel's 2% miss rate is the strongest human result in the comparison set, so it is the most trustworthy estimate of what people can currently achieve. The 5% and 10% figures are not suitable benchmarks because a better human team has already reached 2%. For that reason, 2% is a sensible stand-in for both the estimated benchmark and the desired target. The actual ideal could still be lower than 2%, but it cannot be higher if the panel has already shown 2% performance.

Key points:

  • 2% is the best demonstrated human result.
  • 5% and 10% are weaker than the available human baseline.
  • 2% can serve as both the benchmark estimate and the target.
  • The true ideal may be below 2%, but not above it.

Rubric:

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI