A test set should be big enough to give a confident estimate of the final performance of the system.

Google

A test set is used to evaluate algorithm performance, but not to make decisions about which learning algorithm or parameters to use.

Test Set

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Using 30% of the data for the test set had been a popular heuristic, and it works well with a modest number of examples, such as 100 to 10,000 examples. For big-data machine learning problems, the fraction of data allocated to dev/test sets has been shrinking even as the absolute number of examples in those sets has been growing. Dev/test sets do not need to be excessively large beyond what is needed to evaluate algorithm performance.

70/30 Train/Test Split Heuristic Does Not Apply to Large Datasets

Sizing the Test Set for a Confident Performance Estimate

The test set should be large enough to give high confidence in the overall performance of the system.

Learn Before

Related