Learn Before
Human-Level Error Reference Depends on Current System Error
If a system is currently at 40% error, it does not matter much whether data labels and intuitions come from a junior doctor with 10% error or an experienced doctor with 5% error. If the system is already at 10% error, defining the human-level reference as 2% gives better tools to keep improving the system.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Product Performance Intuition as a Desired Error Rate
Human-Level Error Reference Depends on Current System Error
What is the definition of a 'desired error rate' in Andrew Ng's ML Yearning?
True or False: According to ML Yearning, the desired level of performance can be added to a learning curve.
A desired error rate is a level of error one hopes a learning algorithm will eventually _____.
Match each term to its role in desired error rate analysis as described in ML Yearning.
Order the steps to incorporate a desired error rate into a learning curve, as described in ML Yearning.
According to ML Yearning, to which visualization should the desired level of performance be added?
True or False: A desired error rate reflects the error rate a learning algorithm has already achieved on the dev set.
According to ML Yearning, the desired level of performance should be added to your _____.
Match each element visible on a learning curve to what it communicates to the developer.
Order the reasoning steps that lead a developer to use a desired error rate in learning curve analysis.
Explain how a desired error rate is used and visualized on a learning curve.
Determine how to visualize target performance for a speech project's learning curve.
How should a target error rate be visualized on a learning curve?
Learn After
Why Comparing to Human-Level Performance Helps ML Development
When a system has 40% error, which human-level reference is most useful for guiding improvement?
When a system's error rate is 40%, choosing a junior doctor (10% error) vs. an experienced doctor (5% error) as the human-level reference makes little practical difference.
If your system is already at _____ % error, defining the human-level reference as 2% gives better tools to keep improving.
Match each system error scenario to its implication for selecting a human-level reference.
Order the reasoning steps for deciding whether precision in the human-level reference matters for your system.
Why does a 2% human-level reference give better improvement tools when a system is at 10% error, compared to when it is at 40% error?
At high system error rates such as 40%, a more precise human-level reference always provides significantly better guidance than a less precise one.
According to Machine Learning Yearning, human error rate can be used as the _____ error rate when hoping for human-level performance.
Match each annotator type or system scenario to its description from Machine Learning Yearning.
Order the steps in the decision process for selecting an appropriate human-level error reference for an ML system.