Learn Before
Financial motivation for scaling dev sets beyond 10,000 examples.
Question: In the context of mature machine learning applications, what specific business factor justifies the cost of building a dev set much larger than 10,000 examples?
Sample answer: The direct impact on company profits justifies the cost, as a larger dev set is required to detect the tiny accuracy improvements (like 0.01%) that drive revenue in mature systems.
Key points:
- Direct impact on company profits.
- Need to detect even a 0.01% improvement.
Rubric: The answer must mention company profits and the ability to detect very small accuracy improvements.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Why might a team building a high-value application need a dev set much larger than 10,000 examples?
Teams working on mature, high-value applications like advertising may actively pursue accuracy gains as small as 0.01%.
For high-value applications, the dev set may need to be much larger than _____ examples to detect very small improvements.
Match each high-value application domain to the reason why tiny accuracy improvements matter there.
Order the steps for deciding whether a standard 10,000-example dev set is sufficient for a high-value application.
According to Ng, why does a 0.01% accuracy improvement matter in domains like advertising or web search?
A dev set of exactly 10,000 examples is always large enough to detect accuracy improvements relevant to high-value applications like web search.
In high-value applications, teams may be motivated to find even a _____ improvement because it directly affects the company's profits.
Match each concept to its correct role in the reasoning for larger dev sets in high-value applications.
Arrange the reasoning steps Ng uses to justify larger dev sets in high-value applications, in the order he presents them.
Evaluate the need for ultra-large dev sets in mature business applications.
Sizing a dev set for a high-revenue product recommendation engine.
Financial motivation for scaling dev sets beyond 10,000 examples.