Question: A team is building a machine learning system for two possible projects: a toy app that predicts movie genres and a clinical tool that flags abnormal lab results. Explain why there is no universal formula for how much effort should go into creating dev and test sets, and describe what a team should consider when making that decision.

Sample answer: There is no single formula because the right amount of effort depends on the project’s goals, risk, data, and budget. A clinical tool needs careful evaluation because mistakes can be costly, so the team may need larger and more representative dev and test sets. A low-stakes prototype can often use smaller sets because the cost of collecting more data may outweigh the benefit. Teams should think about how reliable the evaluation must be, how much data is available, how expensive it is to label more examples, and how closely the sets need to match the data the system will face after deployment.

Key points:

• There is no universal rule for dev and test set investment.
• The right choice depends on project stakes, available data, and labeling cost.
• Teams should balance evaluation quality against the resources required to build the sets.

Rubric: The essay should explain that dev and test set planning is context-dependent, mention at least one factor such as project risk, data availability, or labeling cost, and describe the trade-off between spending more resources and obtaining more trustworthy evaluation.