Essay

Using human performance as a reference in machine learning work

Question: Why is it useful to compare a model with human performance when the task is one people can do reliably? How does that help a team decide what to improve first?

Sample answer: If a machine learning system is solving a task that trained people can also solve, the team can measure the model against a human result. That comparison gives a meaningful baseline for error analysis. Instead of guessing where the system is weak, the team can separate the remaining mistakes into clearer categories and see which ones matter most. As a result, the team can focus effort on the biggest sources of error rather than spending time on less important details.

Key points:

  • Tasks that humans can perform provide a practical benchmark for model performance.
  • A human baseline makes it easier to judge whether the system is close to acceptable quality.
  • Error analysis becomes more informative because the team has a concrete reference point.
  • The comparison helps reveal the most important failure modes.
  • That makes it easier to choose the highest-value improvements first.

Rubric: The essay should explain that a human-solvable task provides a human performance benchmark. It should then connect that benchmark to stronger error analysis and finish by showing how this helps the team prioritize improvements more effectively.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI