Case context: A machine learning team is developing an application that needs to perform well across four distinct geographic regions. However, they are considering building their dev set using readily available data from only two of those regions for convenience.

Question: If the team proceeds with a dev set comprising data from only two regions, what will their optimization efforts focus on, and why? What must they do to align their efforts with their true deployment goal?

Sample answer: If the team uses a dev set from only two regions, their optimization efforts will focus solely on improving performance in those two regions, because a team's primary focus inherently becomes improving dev set performance. To align with their true goal of doing well across all four geographies, they must ensure their dev set reflects that broader task by including representative data from all four regions.

Key points:

• Optimization naturally centers entirely on the existing dev set.
• The proposed dev set leads to ignoring two critical target regions.
• The dev set must be expanded to include data from all four regions to reflect the true task.

Rubric: The response should state that the team will end up optimizing only for the two regions present in the dev set, and recommend redesigning the dev set to include data from all four target regions.