Question: In a concise analytical response, explain why a development subset should be drawn from the same population as training data while still being excluded from model fitting.

Sample answer: Using the same source population makes the development subset a fair benchmark for comparing model versions, because it reflects the same kind of examples the training data came from. Keeping it out of the fitting process preserves its value as an independent check on progress, hyperparameter choices, and design decisions. For that reason, it does not need to be large; it only needs enough examples to give a dependable signal for evaluation.

Key points:

• Sampled from the same population as the training data
• Kept separate from model fitting
• Used to compare models and monitor improvement
• Usually smaller than the full training set
• Needs only enough data for reliable evaluation

Rubric: A strong response explains that matching the training population makes comparisons meaningful, states that the subset is held out from fitting, connects that separation to evaluation and progress monitoring, and notes that its size is based on assessment needs rather than training needs.