Learn Before
Purpose of Dev and Test Sets
Once a dev set and test set are defined, a team can try many ideas, such as different learning algorithm parameters, to see what works best. The dev and test sets allow the team to quickly see how well the algorithm is doing, and their purpose is to direct the team toward the most important changes to make to the machine learning system. Measuring each idea's performance on the dev set lets the team quickly decide whether they are heading in the right direction, and helps them identify which ideas to keep refining and which ones to discard.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Purpose of Dev and Test Sets
Choosing Dev and Test Sets to Reflect Future Data
Choosing Dev and Test Sets from the Same Distribution When Possible
Changing Dev/Test Sets or Evaluation Metrics During a Project Is Common
Quickly Establishing Dev/Test Sets and Metric for New Applications
Sizing the Dev Set to Detect Meaningful Accuracy Changes
What is the primary purpose of a development (dev) set in a machine learning project?
The development set is sometimes called the 'hold-out cross validation set'.
The development set is sometimes also called the _____ cross validation set.
Match each development set role to its description in Machine Learning Yearning.
Order the steps for using a dev set to choose between two model configurations.
Which of the following is NOT a stated use of the development set in Machine Learning Yearning?
The dev set and the training set refer to the same data partition in Machine Learning Yearning.
The dev set is used to tune parameters, _____ features, and make other decisions about the learning algorithm.
Match each dev set use from Machine Learning Yearning to the activity that exemplifies it.
Arrange the stages of a machine learning project that involve the dev set in the correct workflow order.
Explain the role and alternative terminology for the development set.
Determine the correct data partition for tuning a model's parameters and features.
List the three primary uses of the development set.
Learn After
Single-Number Evaluation Metric
Initial Dev/Test Set and Metric Helps Quick Iteration
Changing Dev/Test Sets or the Metric When They No Longer Guide the Team
Error Analysis
Double-Checking Labels of Both Misclassified and Correctly Classified Dev Examples
What is the primary purpose of the dev and test sets once they are defined?
True or False: Measuring an idea's performance on the dev set lets a team quickly decide if it is heading in the right direction.
The dev and test sets allow a team to quickly see how well their _____ is doing.
Match each action related to dev/test sets with its described effect.
Order the steps of using dev/test sets to guide iteration on a machine learning idea.
Explain why having dev/test sets and a metric is important for a team's workflow.
A team tries several algorithm changes without a dev set. Diagnose the resulting problem.
In one to three sentences, describe how the dev set helps a team decide which ideas to keep refining.
In one to three sentences, describe how the dev set helps a team decide which ideas to keep refining.
What kind of things might a team try after defining a dev set and test set?
What kind of things might a team try after defining a dev set and test set?
True or False: Having a dev set and metric allows a team to quickly detect which ideas give small or large improvements.
True or False: Having a dev set and metric allows a team to quickly detect which ideas give small or large improvements.