Learn Before
What is the main role of a black-box validation set in model development?
Consider a customer-support classifier that will eventually be tested in an environment where production logs are not fully available. What should a separate held-out set be used for during development?
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What is the main role of a black-box validation set in model development?
True or False: The Blackbox dev set is meant for manually checking individual examples.
An evaluation set that is kept out of manual review during development is often called a _____ set.
Which is an appropriate use of a Blackbox dev set?
A black-box development set should be used only for automated evaluation, not for manual inspection.
A validation split example
Match each evaluation-set term to its definition.
Order the steps for splitting a validation pool into a review set and a machine-only set.
How large is a Blackbox dev set often recommended for many tasks?
A blackbox dev set is evaluated only by automatic metrics, not by reviewing individual examples.
A validation set can be used to _____ candidate algorithms before deployment.
Match each valid purpose of an automated holdout set to the way it is used.
Order the steps that explain why an untouched hidden validation set should stay hidden.
Describe what a blackbox dev set is and how it should be used.
Find the mistake in how a team uses its split evaluation set.
Why is this evaluation subset called the black-box set?