Case context: A team is building a model that will be deployed across five retail warehouses. For convenience, they consider creating the development set from data taken from only two warehouses.

Question: If the development set includes samples from only two warehouses, what will the team end up improving, and why? What change should they make so that their optimization matches the actual deployment goal?

Sample answer: If the development set is built from only two warehouses, the team will mainly optimize performance on those two locations, because model-building decisions naturally track the development set. That means progress on the other three warehouses may be ignored even though they matter in production. To make the optimization objective match the real task, the development set should be redesigned to include representative data from all five warehouses.

Key points:

• The model-development process follows the development set.
• A narrow development set narrows the optimization target.
• The missing warehouses would not be properly represented.
• The development set should reflect the full deployment environment.

Rubric: The response should state that optimization will be concentrated on the two warehouses included in the development set, and it should recommend revising the development set so that it includes representative data from all five warehouses.