Case context: A team is building a mobile app that reads handwritten delivery forms. To speed up early experiments, they created their development set from flatbed-scanner images, but they evaluate the final system with photos taken by drivers on their phones during deliveries. The team is seeing slow improvement and unstable metrics.

Question: Based on good practice for a specific machine learning application, identify what is wrong with the team's current dataset strategy and explain how they should change it to improve development speed.

Sample answer: The development set and test set come from different data distributions: scanned images for development and phone photos for testing. When a team is working on one concrete application, this mismatch makes it harder to judge progress and slows iteration. The better strategy is to make both sets match the data the product will actually see in use, which in this case means using phone photos for both development and test evaluation. Handling one system that must work across multiple distributions is a different research problem and should not be the default choice when the goal is efficient product development.

Key points:

• The current development and test sets are drawn from different distributions
• This mismatch reduces the team's ability to make fast, reliable progress
• Both sets should reflect the same real-world data source, here the phone photos
• Matching the deployment distribution improves iteration speed and metric consistency

Rubric: The answer should recognize that the development and test sets are from different distributions, explain that this slows progress on the application, and recommend using the same distribution for both sets, specifically the phone photos.