Learn Before
Quickly Establishing Dev/Test Sets and Metric for New Applications
For a new project, choosing dev/test sets quickly gives the team a well-defined target. A practical starting point is to create initial dev/test sets and an initial metric in less than one week, even if they are imperfect. This one-week timeline does not apply to mature applications; teams working on already-mature systems may spend months acquiring better dev/test sets.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Purpose of Dev and Test Sets
Choosing Dev and Test Sets to Reflect Future Data
Choosing Dev and Test Sets from the Same Distribution When Possible
Changing Dev/Test Sets or Evaluation Metrics During a Project Is Common
Quickly Establishing Dev/Test Sets and Metric for New Applications
Sizing the Dev Set to Detect Meaningful Accuracy Changes
What is the primary purpose of a development (dev) set in a machine learning project?
The development set is sometimes called the 'hold-out cross validation set'.
The development set is sometimes also called the _____ cross validation set.
Match each development set role to its description in Machine Learning Yearning.
Order the steps for using a dev set to choose between two model configurations.
Which of the following is NOT a stated use of the development set in Machine Learning Yearning?
The dev set and the training set refer to the same data partition in Machine Learning Yearning.
The dev set is used to tune parameters, _____ features, and make other decisions about the learning algorithm.
Match each dev set use from Machine Learning Yearning to the activity that exemplifies it.
Arrange the stages of a machine learning project that involve the dev set in the correct workflow order.
Explain the role and alternative terminology for the development set.
Determine the correct data partition for tuning a model's parameters and features.
List the three primary uses of the development set.
Learn After
For a brand new application, roughly how quickly should a team establish initial dev/test sets and a metric?
True or False: The one-week timeline for establishing dev/test sets applies equally to mature applications.
When starting a new project, quickly choosing dev/test sets gives the team a well-defined _____ to aim for.
Match each project situation to its recommended dev/test set timeline.
Order the steps a team should follow when quickly establishing dev/test sets for a new application.
Why does establishing dev/test sets quickly benefit a team working on a new application?
A team debates spending three weeks perfecting dev/test sets for a brand new application before starting development.
In one to two sentences, state the recommended timeline for creating an initial dev/test set and metric on a new project.
What is the main reason to quickly choose dev/test sets when starting a new project?
True or False: It is acceptable for the initial dev/test set and metric on a new project to be imperfect.