Case context: A startup is building a spam filter for customer emails. The engineers keep changing the feature set, training settings, and network design, but they never reserved a development set or a test set. After many rounds of experimentation, they cannot agree on whether the newest version is actually better.

Question: Using the role of development and test sets, explain what the team is missing and how adding those sets would help them settle the disagreement.

Sample answer: The team is missing a development set and a test set, which provide a quick and objective way to check whether a change improves the system. Without them, the engineers have no reliable reference for judging each experiment, so opinions replace measurement. If they create these sets, they can compare ideas on the development set, see which changes help, and use the test set for a final unbiased check before releasing the model.

Key points:

• The project lacks development and test sets for objective evaluation
• Without them, the team cannot judge experiments in a consistent way
• A development set lets the team compare changes and track improvement
• A test set gives a final unbiased check before deployment

Rubric: Full credit: identifies the missing development and test sets as the reason the team cannot evaluate changes reliably, and explains that these sets enable quick comparison and a final objective check. Partial credit: mentions the missing development set but gives little explanation of its benefit. No credit: does not connect the disagreement to the purpose of development and test sets.