Question: Write a short essay explaining why a 70%/30% random split can give a misleading evaluation when the data the model is trained on is not drawn from the same conditions where it will later be used.

Sample answer: A random split divides only the examples that are already available, so it keeps the same mix of patterns, noise, and biases found in that pool of data. If the available examples come from one setting, but the model must work in a different setting, the test set will still reflect the first setting rather than the future one. For example, a model trained on product photos taken in a studio may be tested on a random holdout from those studio photos, even though the real job is to recognize items in dim, handheld snapshots from customers. In that case, the score on the holdout set can look good while saying little about how the model will behave in the customer setting. That is why the split is not a reliable check of real-world performance.

Key points:

• A random split mirrors the distribution of the data already collected
• The collected data can differ from the deployment environment
• Example: studio product photos versus handheld customer photos
• The holdout set may stay representative of the source pool, not the target use case
• The evaluation can therefore give an overly optimistic or otherwise misleading estimate of deployment performance

Rubric: Full credit requires stating that a random split preserves the source-data distribution, identifying a concrete source-versus-target mismatch, and explaining that the resulting test score may not reflect performance in the environment where the model will actually be used.