Case context: A logistics company is building a classifier that sorts package images into shipping categories. Each time the team trains a new model, they do not have a fixed dev set or a clear evaluation metric. Instead, they load the model into a demo app, point a tablet at a pile of sample labels, and judge by eye whether the new version seems better. They want a more reliable workflow.

Question: Based on the recommended machine learning workflow, what is the main weakness of the team's current evaluation process? What should they put in place, and how will that change how they choose which ideas to pursue?

Sample answer: The current process is too slow and too subjective, so it is hard to notice small improvements. The team should create a specific development set and a clear evaluation metric. With those in place, they can compare model versions quickly and quantitatively, which makes it easier to decide whether a new idea is worth refining or should be dropped.

Key points:

• Identify manual evaluation as slow and subjective.
• Recommend a specific development set and evaluation metric.
• Explain that these tools enable fast, quantitative comparison so the team can keep promising ideas and discard weak ones.

Rubric: The response must identify the slowness and subjectivity of manual testing. It must recommend a specific development set and evaluation metric. It must explain that this setup allows quick quantitative comparisons that guide which ideas to refine or discard.