Learn Before
Explain the boundary between evaluating with a test set and making decisions from it.
Question: In a concise analytical response, explain what a test set should be used for and which decisions it should not influence.
Sample answer: A test set should be used to evaluate the performance of an algorithm. It should not be used to decide which learning algorithm to use or which parameters to select. Thus, its role is evaluation rather than algorithm or parameter decision-making.
Key points:
- The test set evaluates algorithm performance.
- It does not determine which learning algorithm to use.
- It does not determine which parameters to use.
- Evaluation and selection are distinct roles.
Rubric: A strong response clearly states the evaluation role, excludes both learning-algorithm selection and parameter selection, and accurately distinguishes evaluation from decision-making.
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Machine Learning
Deep Learning
Supervised Learning
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
70/30 Train/Test Split Heuristic Does Not Apply to Large Datasets
Sizing the Test Set for a Confident Performance Estimate
Test Set Should Support High-Confidence Performance Estimates
Which use of a test set is consistent with its stated purpose?
A test set may evaluate an algorithm without helping select its parameters.
The test set is used to _____ algorithm performance.
Match each test-set concept with its correct role.
Order the reasoning for deciding whether a proposed test-set use is appropriate.
Explain the boundary between evaluating with a test set and making decisions from it.
Diagnose a team that uses test results to choose an algorithm and its parameters.
What is the test set's role, and what two decisions must it not guide?
Which proposal misuses the test set?
Test-set performance should determine which learning algorithm is used.