Learn Before
Choosing the right split for model selection decisions.
Case context: A hospital analytics group is developing a model to predict whether patients will return within 30 days. They already have a labeled dataset and want to decide which input variables to keep and which model settings to adjust.
Question: Which dataset split should they use to compare modeling choices and make those selections, and what is that split also called?
Sample answer: They should evaluate options on the development set, which is also known as the validation set or hold-out cross-validation set, because it is used for choosing features and tuning model settings.
Key points:
- Names the development set as the correct split.
- Gives another acceptable name: hold-out cross-validation set.
- Explains that feature selection and parameter tuning are performed using this split.
Rubric: The answer should identify the development set and state its alternative name as the appropriate split for feature selection and parameter tuning.
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Machine Learning
Deep Learning
Supervised Learning
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Dev and Test Sets Matter
How to Select Dev and Test Data for the Future Task
Choose Dev and Test Sets to Match the Main Goal When Feasible
Revise Evaluation Data and Metrics When They Stop Serving the Goal
Set the First Development and Test Splits Early for a New Project
Choosing a Dev Set Large Enough to See Small Gains
What is the main role of a development set during model iteration?
Another Name for the Development Set
Another name for the development set
Match each development-split role to its description.
Order the steps for choosing between two candidate spam filters using a validation set.
Which option is NOT a typical purpose of a development set?
The validation set is used to update model weights in the same way as the training set.
The validation set is used to tune parameters, _____ features, and make other choices about the learning method.
Match each development-set use to the task that illustrates it.
Arrange the main stages of a project that uses a development set.
What is the development set used for in model building?
Choosing the right split for model selection decisions.
What are the main roles of a development set?