Question: Write a short essay explaining why a strong mismatch between a manually reviewed validation set and an untouched validation set suggests overfitting, and why expanding the manually reviewed set can help.

Sample answer: If the system looks much better on the untouched validation set than on the manually reviewed set, that often means the team has been adjusting decisions based too heavily on a small collection of reviewed cases. In that situation, the reviewed set stops acting like a broad checkpoint and starts reflecting the quirks of the few examples that were repeatedly inspected.

The untouched validation set is useful because it is not being used to steer those manual adjustments, so it gives a cleaner picture of how the model is likely to perform on new data. When the reviewed set is enlarged, each individual example matters less, unusual cases have less influence, and the set becomes more representative of the full problem. That makes it harder to tailor choices to a tiny sample and reduces the chance of overfitting to the reviewed cases.

Key points:

• Repeated human inspection of a small reviewed set can cause decisions to fit that set too closely
• The untouched validation set provides a less biased estimate of general performance
• A large difference between the two sets is a warning sign for overfitting
• Adding more reviewed examples lowers the weight of any single case
• A larger reviewed set is usually more representative and less fragile

Rubric: Full credit requires identifying that the gap comes from excessive manual tuning on a small set, explaining that the untouched set serves as an unbiased performance check, and describing how increasing the reviewed set size reduces the influence of individual examples and improves representativeness.