Learn Before
Choosing a Quarterly Target from Years of Operational Experience
Case context: A company has maintained a medical transcription service for many years. The engineering team is now deciding what accuracy target to set for the next release cycle.
Question: According to the underlying idea, how should the team choose the target, and what kind of judgment should guide it?
Sample answer: The team should use its long history with this system to estimate what improvement is realistically attainable in the next cycle. Because they have seen the system’s behavior over many releases, they can make a practical judgment about likely progress, and that judgment should be used to choose the target accuracy.
Key points:
- Years of experience with one application can support realistic forecasting
- The team should estimate how much improvement is feasible in the coming cycle
- That estimate should shape the target they set
- This is about operational performance, not about user-facing product taste
Rubric: Full credit for explaining that long-term experience with the system should be used to judge realistic near-term improvement, and that this judgment should determine the target accuracy.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What kind of reference is most useful for choosing a target error rate in a user-facing classifier?
True or False: Long experience with a product can help you judge what improvement is realistic over the next few months or year.
A product team may care about the level of _____ the model must achieve to deliver a strong user experience.
Match each situation to the kind of intuition it uses when choosing a target error rate.
Order the steps for setting a target error rate in a hospital triage classifier.
How can product experience guide a target error rate?
Choosing a Quarterly Target from Years of Operational Experience
Identify Two Inputs for Setting a Target Error Rate
What Experience Usually Builds Good Improvement Intuition?
True or False: A reasonable target error rate can only be estimated from experience with a consumer-facing product.