1Cademy - When should post-launch user data be used to refresh dev and test sets?

Learn Before

Refreshing Evaluation Sets After a Product Launch

Short Answer

When should post-launch user data be used to refresh dev and test sets?

Question: Answer in one to three sentences: After a product has shipped, when is it appropriate to use real user data to revise the dev and test sets, and which sets may be refreshed?

Sample answer: Once the app is live and you have collected real user behavior, you may rebuild or update both the development set and the test set.

Key points:

After launch
Using real user data
Updating both development and test sets

Rubric: The answer must mention that the app is already launched, that the data comes from actual users, and that both the dev set and the test set can be updated.

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Machine Learning Strategy

Supervised Learning