Benefit of a precise human-level reference for a low-error ML system.
Question: If your machine learning system is already performing well at a 10% error rate, what is the primary benefit of defining your human-level reference as 2% rather than settling for a less precise reference?
Sample answer: Defining a tighter human-level reference of 2% when the system is already at 10% error provides you with better tools to keep improving the system.
Key points:
- A precise human-level reference is needed when system error is already low.
- It provides better tools to keep improving the system.
Rubric: The answer must mention that a more precise human-level reference provides better tools or guidance to continue improving an already well-performing system.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Comparing to Human-Level Performance Helps ML Development
When a system has 40% error, which human-level reference is most useful for guiding improvement?
When a system's error rate is 40%, choosing a junior doctor (10% error) vs. an experienced doctor (5% error) as the human-level reference makes little practical difference.
If your system is already at _____ % error, defining the human-level reference as 2% gives better tools to keep improving.
Match each system error scenario to its implication for selecting a human-level reference.
Order the reasoning steps for deciding whether precision in the human-level reference matters for your system.
Why does a 2% human-level reference give better improvement tools when a system is at 10% error, compared to when it is at 40% error?
At high system error rates such as 40%, a more precise human-level reference always provides significantly better guidance than a less precise one.
According to Machine Learning Yearning, human error rate can be used as the _____ error rate when hoping for human-level performance.
Match each annotator type or system scenario to its description from Machine Learning Yearning.
Order the steps in the decision process for selecting an appropriate human-level error reference for an ML system.
Explain the relationship between current system error and the choice of human-level reference.
Choose the appropriate medical expert reference for a system with 40% error.
Benefit of a precise human-level reference for a low-error ML system.