Prioritizing Scoring Improvements from Optimization Verification Counts
If Optimization Verification shows that 95% of errors are due to the scoring function and only 5% are due to optimization, improving the optimization procedure can realistically eliminate only about 5% of errors. The better focus is then improving the score estimation.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Prioritizing Scoring Improvements from Optimization Verification Counts
When applying the Optimization Verification test to a dev set error, which condition marks the error as caused by the optimization algorithm?
True or False: When Score_A(S*) ≤ Score_A(S_out) for a dev set error, the error is attributed to the optimization algorithm.
Each dev set error where Score_A(S*) ≤ Score_A(S_out) is counted as a mistake due to the _____.
Which condition classifies a dev set error as an optimization-algorithm error in the Optimization Verification test?
The Optimization Verification test should be applied to every error in the dev set, not just a single example.
When Score_A(S*) ≤ Score_A(S_out), the dev set error is classified as a _____ error.
Match each Optimization Verification inequality or symbol to its correct interpretation.
Order the steps for applying the Optimization Verification test across all errors in the dev set.
What does the condition Score_A(S*) ≤ Score_A(S_out) reveal about a dev set error?
Andrew Ng's Optimization Verification discussion initially focused on a single example before scaling the approach to the full dev set.
In the Optimization Verification test, S* represents the _____ output for a given input.
Match each Optimization Verification concept to its role in diagnosing dev set errors.
Order the decision-making steps after tallying Optimization Verification results across the dev set.
How should you partition dev set errors using the Optimization Verification test?
Diagnosing Dev Set Errors Using Optimization Verification Inequalities
What condition identifies a scoring-function error in optimization verification?
Learn After
Optimization Verification finds 95% of errors trace to the scoring function and 5% to the optimization algorithm. Where should effort go?
If 5% of errors are due to the optimization algorithm, perfecting it could eliminate at most about 5% of total errors.
When 95% of errors are due to the scoring function, you should focus on improving how you _____ the score.
Match each error share from Optimization Verification to its correct implication for improvement effort.
Order the reasoning steps for deciding where to focus improvement effort after Optimization Verification.
Explain why an error attribution of 95% scoring function versus 5% optimization algorithm should redirect a team's improvement priorities.
A team must decide where to invest engineering time after running Optimization Verification.
In one to three sentences, explain what upper bound a 5% optimization-algorithm error share places on possible improvements.
Which statement correctly describes how to prioritize improvements based on Optimization Verification error percentages?
Optimization Verification error percentages tell a team the maximum error reduction achievable by improving each component.