Learn Before
Detecting Small Performance Improvements with a Dev Set and Metric
Without a specific dev set and metric, a team may have to incorporate a new classifier into an app and manually try the app to judge whether the classifier is an improvement. Having a dev set and metric allows the team to quickly detect which ideas are giving small or large improvements, and therefore quickly decide which ideas to keep refining and which ones to discard.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Learn After
Without a dev set and metric, how must a team evaluate whether a new classifier is an improvement?
A dev set and metric allows a team to quickly detect whether new classifier ideas produce small or large improvements.
A dev set and metric lets teams quickly decide which ideas to keep _____ and which ones to discard.
Match each situation to its consequence when evaluating a new classifier.
Order the steps a team must take to evaluate a new classifier when NO dev set or metric exists.
What does having both a dev set and metric enable a team to do that manual app testing does not?
According to Ng, manually testing each new classifier by playing with the app is a fast, efficient evaluation method.
Without a dev set and metric, each time a team develops a new classifier, they must _____ it into the app to evaluate it.
Match each concept to its role in the classifier evaluation process described by Ng.
Order the steps a team follows when using a dev set and metric to evaluate and iterate on classifier ideas.
Analyzing the Efficiency of Dev Sets and Metrics vs. Manual App Testing
Evaluating Classifier Iterations for a Mobile Application
Contrast of Classifier Refinement Decisions