Learn Before
Why Error Percentages Alone Do Not Determine Task Priority
Question: When a machine learning team is deciding what to fix next, why is the share of errors in each category not enough to choose the top priority? What other considerations should guide the decision?
Sample answer: Error counts show how often each problem appears, but frequency by itself does not tell you what to fix first. Some categories are straightforward to improve and can produce a large gain, while others are costly to address or have only a small upside. A good prioritization also considers the likely improvement available in each category and the amount of time, engineering effort, or data work required to achieve that improvement. In practice, the best next task is often the one with a strong balance of impact and feasibility, not simply the most common error.
Key points:
- Error percentages alone are not enough for prioritization
- You must estimate the likely improvement available in each category
- You must weigh the work, time, and resources needed to fix each category
Rubric: A strong response should explain that frequency does not equal priority. It must mention both the expected benefit from fixing a category and the effort required to do so.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does error analysis not directly give you when deciding the next improvement to make?
Error analysis alone can identify the single highest-priority improvement with no further judgment.
Error analysis does not provide a fixed priority rule
Match each review metric to its role in deciding what to fix first.
Prioritize work using error analysis.
Which two factors should be considered in addition to error percentages when choosing what to fix first?
When deciding which error categories to address first, the effort required for each fix should be considered.
Tradeoffs in Choosing Which Problems to Fix First
Match each description to the role it plays in deciding which error to fix first.
Order the steps for deciding whether one error category should be prioritized first.
Why Error Percentages Alone Do Not Determine Task Priority
Choosing between two fixes after review of failure patterns.
Other factors in choosing which error type to fix first