How can post-launch user data change evaluation sets in an app project?
Question: In a brief analytical response, explain what new option becomes available after a mobile app is released and identify the data source and the evaluation sets it can affect.
Sample answer: Once the app is in users’ hands, the team can use real usage data to revise the development and evaluation sets. In other words, release makes it possible to refresh those sets with examples from actual users instead of keeping the original splits unchanged forever.
Key points:
- The app has already been released.
- Real user data becomes available.
- The development set can be revised.
- The evaluation/test set can be revised.
Rubric: A strong response should mention the post-release timing, name real user data as the source for updates, and identify both the development set and the test set as changeable. Do not award credit for unrelated claims about model training procedures or performance guarantees.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Refreshing evaluation sets after deployment
True or False: Evaluation sets must remain frozen forever after deployment.
After deployment, evaluation sets may be refreshed with actual _____ data.
Match the post-release data update components
Order the steps for refreshing evaluation data after launch
How can post-launch user data change evaluation sets in an app project?
Case Study: Updating evaluation data after a product launch
When should post-launch user data be used to refresh dev and test sets?
A logistics app is launched and begins collecting live customer records. Which action is supported?
Real user logs can be used to refresh both evaluation sets after launch.