Blackbox Dev Set
The Blackbox dev set is the second subset of a split dev set, consisting of the remaining examples after the Eyeball dev set is drawn; in the running example it contains 4,500 examples. It can be used to evaluate classifiers automatically by measuring their error rates, to select among algorithms, and to tune hyperparameters. One should avoid looking at it with one's eyes. The term Blackbox is used because this subset is only used to obtain blackbox evaluations of classifiers.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Learn After
Which of the following best describes the intended use of the Blackbox dev set?
True or False: It is appropriate to manually examine individual examples in the Blackbox dev set.
The Blackbox dev set is named after the _____ evaluations it provides, since examples in it are never manually inspected.
Which of the following is a valid use of the Blackbox dev set?
You should avoid looking at the Blackbox dev set with your eyes.
In the running example from Machine Learning Yearning, the Blackbox dev set contains _____ examples.
Match each dev set term to its defining characteristic.
Order the steps for setting up and using a split dev set with both Eyeball and Blackbox subsets.
What size range for a Blackbox dev set does Machine Learning Yearning say is sufficient for many applications?
The term 'Blackbox' is used because this dev set subset is used only for automated evaluations without manually examining examples.
The Blackbox dev set can be used to _____ among algorithms when comparing different model approaches.
Match each permitted use of the Blackbox dev set to a description of how that use is carried out.
Order the reasoning steps that justify keeping the Blackbox dev set off-limits for manual inspection.
Explain the core purpose and proper handling of a Blackbox dev set.
Diagnose a methodology error in a team's use of a split dev set.
Explain the origin of the term 'Blackbox' dev set.