Learn Before
70/30 Train/Test Split Heuristic Does Not Apply to Large Datasets
Using 30% of the data for the test set had been a popular heuristic, and it works well with a modest number of examples, such as 100 to 10,000 examples. For big-data machine learning problems, the fraction of data allocated to dev/test sets has been shrinking even as the absolute number of examples in those sets has been growing. Dev/test sets do not need to be excessively large beyond what is needed to evaluate algorithm performance.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
70/30 Train/Test Split Heuristic Does Not Apply to Large Datasets
Sizing the Test Set for a Confident Performance Estimate
Test Set Should Support High-Confidence Performance Estimates
Which use of a test set is consistent with its stated purpose?
A test set may evaluate an algorithm without helping select its parameters.
The test set is used to _____ algorithm performance.
Match each test-set concept with its correct role.
Order the reasoning for deciding whether a proposed test-set use is appropriate.
Explain the boundary between evaluating with a test set and making decisions from it.
Diagnose a team that uses test results to choose an algorithm and its parameters.
What is the test set's role, and what two decisions must it not guide?
Which proposal misuses the test set?
Test-set performance should determine which learning algorithm is used.
Learn After
For which dataset size is the 70/30 train/test split heuristic most appropriate?
True or False: As ML datasets grow to billions of examples, the fraction of data allocated to dev/test sets should grow proportionally to maintain the 70/30 split.
For large-scale ML problems, the _____ of data allocated to dev/test sets has been shrinking even as the absolute number of examples grows.
For which example-count range does the 70/30 train/test split heuristic work well, according to Andrew Ng?
As total dataset size grows into the billions, the fraction of data allocated to dev/test sets also increases.
The 70/30 train/test heuristic works well when you have a _____ number of examples, such as 100 to 10,000.
Match each dataset scale or concept to the correct dev/test sizing implication from Machine Learning Yearning.
Order the reasoning steps Ng uses to argue that the 70/30 heuristic does not apply to large datasets.
In big-data ML problems, what happens to the absolute number of dev/test examples as total dataset size grows?
Dev/test sets should be no larger than what is needed to evaluate the performance of your algorithms.
One popular heuristic was to use _____ of your data for your test set, though this does not apply to large datasets.
Match each train/dev/test splitting term to its correct definition from Machine Learning Yearning.
Order the steps a practitioner should follow when deciding test set size as their dataset grows from thousands to billions of examples.
Why does the traditional 70/30 train/test split heuristic fail to scale efficiently for big-data machine learning problems?
Evaluating dev and test set sizes for a billion-scale image classification system
Determining dev/test set size in the era of big-data machine learning