Learn Before
Human-Solvable Problems Enable More Powerful Error Analysis Tools
Many error analysis processes work best when the system automates something humans can do, because human-level performance can be used as a benchmark. If the final output or intermediate components do things humans cannot do well, some procedures do not apply.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Examples Can Belong to Multiple Error Categories
Discovering New Error Categories While Reviewing Examples
Most Helpful Error Categories Are Ones You Have Ideas to Improve
Error Analysis Is an Iterative Process
Error Category Frequency Helps Indicate Which Categories to Focus On
Pursuing Multiple Error Categories in Parallel
Error Analysis Does Not Yield a Rigid Priority Formula
Error Category Fraction as a Ceiling on Possible Error Reduction
Error Analysis as a Quantitative Basis for Project Investment Decisions
Engineer Reluctance to Perform Error Analysis Despite Its Low Time Cost
Mislabeled Examples in the Dev Set
Splitting a Large Dev Set into a Manually Examined Subset and a Hands-Off Subset
Build a Basic System Quickly and Iterate Using Error Analysis
Training Set Error Analysis for High Bias
Manually Reviewing 100 Speech Recognition Dev Set Examples
Debugging Inference Algorithms
Error Analysis by Parts
Error Analysis as Data Science for ML Mistakes
No Single Right Way to Perform Error Analysis
Human-Solvable Problems Enable More Powerful Error Analysis Tools
What does error analysis primarily examine to understand an ML system's mistakes?
There is exactly one correct method for conducting error analysis on an ML system.
The process of looking at misclassified examples to understand error causes is called _____.
Match each error analysis concept to its correct description from Machine Learning Yearning.
Order the steps of conducting a basic error analysis on a dev set as described in Machine Learning Yearning.
What is the primary goal of reviewing misclassified examples during error analysis, even in categories you cannot yet fix?
Machine Learning Yearning describes error analysis as an iterative process.
Error analysis can often help you figure out how _____ different improvement directions are.
Match each error analysis activity to the benefit it provides according to Machine Learning Yearning.
Order the reasoning steps for deciding which error categories to pursue after completing an initial error analysis.
Learn After
Why do error analysis processes work best when an ML system automates a human-solvable task?
When an ML system automates a task humans can do well, human-level performance can serve as a benchmark for error analysis.
Working on human-solvable problems provides more powerful _____ tools, enabling more efficient team prioritization.
Match each task characteristic to its implication for applying error analysis procedures.
Order the steps for applying human-level benchmarking in error analysis for a human-solvable ML task.
An ML team builds a system performing a task no human expert can reliably evaluate. What should the team expect about error analysis?
All standard error analysis procedures in Machine Learning Yearning apply equally regardless of whether humans can perform the task.
If an ML system's final output or intermediate components do things _____ cannot do well, some error analysis procedures will not apply.
Match each ML development scenario to its consequence for error analysis tooling.
Order the reasoning steps explaining why human-solvable problems lead to more efficient team prioritization in ML development.