Approximating Future Dev/Test Data Before Launch
Before a mobile app has launched, actual user data may not yet be available for dev/test sets. In that case, one can try to approximate the future data, for example by asking friends to send mobile-phone cat pictures.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Related
Adding More Training Data Does Not Always Help
Special Challenges from Different Training and Dev/Test Distributions
Risk of Merging Training Data Sources Depends on Algorithm Flexibility
Shared Label Mapping Across Data Sources
Training and Dev/Test Sets from Different Distributions
Inconsistent Auxiliary Data Source
Approximating Future Dev/Test Data Before Launch
Updating Dev/Test Sets with Actual User Data After Launch
Risk of Starting with Website Images When Future-Like Data Is Unavailable
Development Investment for Dev and Test Sets Requires Judgment
According to Machine Learning Yearning, what is the primary criterion for choosing dev and test sets?
True or False: When building a dev/test set, it is safe to assume the training distribution is the same as the test distribution.
Dev and test sets should contain examples that reflect what you ultimately want to perform well on, rather than only the _____ you happen to have for training.
Why is using a simple 30% random split of available data as your test set problematic when future data differs from training data?
According to ML Yearning, it is generally safe to assume your training data distribution is the same as your test data distribution.
Dev and test sets should be chosen to reflect data you expect to get in the _____ and want to do well on.
Match each dev/test set concept from ML Yearning to its correct description.
Order the steps for correctly choosing dev and test sets according to ML Yearning's guidance.
According to ML Yearning, what should the examples in your dev and test sets primarily reflect?
According to ML Yearning, dev and test sets must always come from the same distribution as the training data.
ML Yearning warns that the test set should not simply be _____ of the available data when future data differs from the training set.
Match each data scenario to the correct dev/test set strategy decision according to ML Yearning.
Order the reasoning steps for deciding whether a proposed dev/test set is well-chosen, per ML Yearning.
Why Standard Data Splits Fail With Different Future Distributions
Dev and Test Set Design for Mobile Image Applications
The Core Criterion for Dev and Test Set Selection
Learn After
Before a mobile app launches, how can you approximate future dev/test data when no user data is available?
Before a mobile app launches, it is impossible to approximate the future data distribution for dev/test sets.
Before launching a mobile app, you can approximate future user data by asking friends to take _____ pictures of cats and send them to you.
When a mobile app has not yet launched and no user data is available, what is the recommended approach for building dev/test sets?
Before a mobile app launches, it is impossible to construct any dev/test set because real user data does not yet exist.
When no user data exists before app launch, Andrew Ng recommends trying to _____ the future data distribution for dev/test sets.
Match each pre-launch data concept to its correct description.
Order the steps a team should follow to approximate pre-launch dev/test data for a mobile app.
Which specific example does Andrew Ng give in Machine Learning Yearning for approximating pre-launch mobile app data for a cat-detection use case?
Data collected from friends before a mobile app's launch perfectly represents the distribution of future user data.
Andrew Ng suggests asking your _____ to take mobile-phone pictures of cats as one way to approximate pre-launch user data.
Match each pre-launch data approximation term to its correct explanation.
Order the reasoning steps for deciding how to handle the absence of user data when constructing dev/test sets before a mobile app launches.
Explain the rationale and challenges of approximating future dev/test data before a mobile application launch.
Case Study: Designing a cat detection dev/test set before mobile application launch.
What is the purpose of asking friends to take photos when constructing a pre-launch dev/test set?