Choose evaluation strategy for a crop disease app.
Case context: A startup is building a mobile system that identifies leaf diseases from photos taken by farmers. The team has a large archive of high-quality greenhouse images and a smaller set of field photos captured in varied lighting, backgrounds, and camera conditions. An engineer suggests spending significant time on cross-domain transfer methods so the model can be trained on greenhouse images and then adapted to field photos.
Question: For the startup's immediate product goals, should the team prioritize that transfer-focused approach? Explain your choice.
Sample answer: No. For near-term product work, the team should make the evaluation data match the real target use case: field photos from farmers. Transfer methods can be useful in some situations, but the first priority is to build dev and test sets from the target population so progress is measured on the data that matters most. That usually helps the team move faster and make clearer decisions about model improvements.
Key points:
- The immediate goal is building a useful product, not exploring a broad research direction.
- The dev and test sets should reflect the field-photo distribution.
- Cross-domain transfer is not the main first step for this situation.
- Matching evaluation data to the target setting improves team efficiency and decision-making.
Rubric: The response should recommend against prioritizing the transfer-focused approach for the immediate goal and instead advise using dev and test sets drawn from the target field-photo distribution to improve team efficiency.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Use Matching Dev and Test Distributions for Application Work
For an application project, what data-splitting rule is most helpful?
Domain adaptation methods are the default solution for most machine learning projects.
Domain adaptation means training on one _____ and expecting useful performance on another.
Match each machine-learning concept to its description.
Order the steps for choosing dev/test data when the project data differs from deployment data.
When are domain adaptation methods usually a good fit?
Selecting development and test sets from the same data distribution can improve team efficiency.
Dev and test sets should come from the _____ distribution.
Match each project goal to the strategy associated with it.
Order the main ideas behind matching dev and test data to the deployment setting.
Domain Adaptation Research Versus Application-Focused Data Splits
Choose evaluation strategy for a crop disease app.
Why choose dev and test sets from the same distribution instead of building a special adaptation method?